Fix a regression in PR345 in transparent compression of event files#459
Conversation
… `userio` module. The resulting file in `events_home` would silently fail during writes for the compressed cases due to a subtle bytes vs str issue for gzip and bz2 file handles, when using mode strings without a specific binary or text marker. They default to open in binary mode in that case, which breaks writing of plain strings with an error like: `memoryview: a bytes-like object is required, not 'str'` To make matters worse our old built-in self-tests did not verify the written event files at all, so when they weren't created it looked like things were alright. Adjusted the gzip and bz2 compression cases to explicitly use text mode for regular string writing to address the regression. Reworked the self-tests to include careful verification of the written event files and their contents as a regression test. It was necessary to expose the event timestamps as an optional arg in the function APIs to enable such testing. Also addressed a minor issue with the clean-up helper not fully cleaning up after 'remove' tests, where the data is really moved into the user Trash folder in the root of the user home. Further modern unit tests of the `userio` module would be nice, but that bit is saved for later.
|
@jonasbardino I don't see the changes to the tests mentioned in the description in the changed files currently. |
They're limited to the old-fashioned inline self-tests at the |
Ohhh my apologies, the tests are in the main method aren't they? I saw the _verify thing but couldn't immediately see where it is called. This is another style of some of the earlier checks that I forgot to look for. I don't think it needs brand new ones, and I can see there is a legacy bridge wrapper that triggers these so they are exercised. I think that is enough. Will attempt to eyeball what's here in more detail next chance I get. |
|
I've started adding modern unit tests for the |
rasmunk
left a comment
There was a problem hiding this comment.
Approved with the expected unittest followup
Fix a regression in the recent addition of transparent compression in
useriomodule (#345). The resulting file inevents_homewould silently fail during writes for the compressed cases due to a subtle bytes vs str issue for gzip and bz2 file handles, when using mode strings without a specific binary or text marker. They default to open in binary mode in that case, which breaks writing of plain strings with an error like:memoryview: a bytes-like object is required, not 'str'To make matters worse our old built-in self-tests did not verify the written event files at all, so when they weren't created it looked like things were alright.
Adjusted the gzip and bz2 compression cases to explicitly use text mode for regular string writing to address the regression.
Reworked the self-tests to include careful verification of the written event files and their contents as a regression test. It was necessary to expose the event timestamps as an optional arg in the function APIs to enable such testing.
Also addressed a minor issue with the clean-up helper not fully cleaning up after 'remove' tests, where the data is really moved into the user Trash folder in the root of the user home.
Further modern unit tests of the
useriomodule would be nice, but that bit is saved for later.