Skip to content

Fix a regression in PR345 in transparent compression of event files#459

Merged
jonasbardino merged 2 commits intonextfrom
fix/byte-vs-string-regression-in-userio-event-auto-compression
Feb 17, 2026
Merged

Fix a regression in PR345 in transparent compression of event files#459
jonasbardino merged 2 commits intonextfrom
fix/byte-vs-string-regression-in-userio-event-auto-compression

Conversation

@jonasbardino
Copy link
Contributor

Fix a regression in the recent addition of transparent compression in userio module (#345). The resulting file in events_home would silently fail during writes for the compressed cases due to a subtle bytes vs str issue for gzip and bz2 file handles, when using mode strings without a specific binary or text marker. They default to open in binary mode in that case, which breaks writing of plain strings with an error like:
memoryview: a bytes-like object is required, not 'str'

To make matters worse our old built-in self-tests did not verify the written event files at all, so when they weren't created it looked like things were alright.

Adjusted the gzip and bz2 compression cases to explicitly use text mode for regular string writing to address the regression.

Reworked the self-tests to include careful verification of the written event files and their contents as a regression test. It was necessary to expose the event timestamps as an optional arg in the function APIs to enable such testing.
Also addressed a minor issue with the clean-up helper not fully cleaning up after 'remove' tests, where the data is really moved into the user Trash folder in the root of the user home.

Further modern unit tests of the userio module would be nice, but that bit is saved for later.

… `userio`

module. The resulting file in `events_home` would silently fail during writes
for the compressed cases due to a subtle bytes vs str issue for gzip and bz2
file handles, when using mode strings without a specific binary or text marker.
They default to open in binary mode in that case, which breaks writing of plain
strings with an error like:
`memoryview: a bytes-like object is required, not 'str'`

To make matters worse our old built-in self-tests did not verify the written
event files at all, so when they weren't created it looked like things were
alright.

Adjusted the gzip and bz2 compression cases to explicitly use text mode for
regular string writing to address the regression.

Reworked the self-tests to include careful verification of the written event
files and their contents as a regression test. It was necessary to expose the
event timestamps as an optional arg in the function APIs to enable such
testing.
Also addressed a minor issue with the clean-up helper not fully cleaning up
after 'remove' tests, where the data is really moved into the user Trash folder
in the root of the user home.

Further modern unit tests of the `userio` module would be nice, but that bit is
saved for later.
@jonasbardino jonasbardino marked this pull request as ready for review February 14, 2026 15:31
@jonasbardino jonasbardino requested a review from a team February 14, 2026 15:31
@albu-diku
Copy link
Contributor

@jonasbardino I don't see the changes to the tests mentioned in the description in the changed files currently.

@jonasbardino
Copy link
Contributor Author

@jonasbardino I don't see the changes to the tests mentioned in the description in the changed files currently.

They're limited to the old-fashioned inline self-tests at the __main__ part of the userio module itself. Hint: __verify_test_events.
I'll see if I can find the time to add modern tests of the entire userio module as hinted above, but that will happen in a standalone PR.

@albu-diku
Copy link
Contributor

albu-diku commented Feb 15, 2026

@jonasbardino I don't see the changes to the tests mentioned in the description in the changed files currently.

They're limited to the old-fashioned inline self-tests at the __main__ part of the userio module itself. Hint: __verify_test_events. I'll see if I can find the time to add modern tests of the entire userio module as hinted above, but that will happen in a standalone PR.

Ohhh my apologies, the tests are in the main method aren't they? I saw the _verify thing but couldn't immediately see where it is called. This is another style of some of the earlier checks that I forgot to look for.

I don't think it needs brand new ones, and I can see there is a legacy bridge wrapper that triggers these so they are exercised. I think that is enough.

Will attempt to eyeball what's here in more detail next chance I get.

@jonasbardino
Copy link
Contributor Author

jonasbardino commented Feb 15, 2026

I've started adding modern unit tests for the userio module in #461 and would like to eventually add with a proper regression test for the issue there.

jonasbardino added a commit that referenced this pull request Feb 15, 2026
regression test for issue #460 to be enabled when fix in #459 is merged.
@rasmunk rasmunk requested review from a team and rasmunk and removed request for a team and rasmunk February 17, 2026 08:23
Copy link
Contributor

@rasmunk rasmunk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with the expected unittest followup

@jonasbardino jonasbardino merged commit 6aecd6a into next Feb 17, 2026
14 checks passed
@jonasbardino jonasbardino deleted the fix/byte-vs-string-regression-in-userio-event-auto-compression branch February 17, 2026 08:39
jonasbardino added a commit that referenced this pull request Feb 24, 2026
regression test for issue #460 to be enabled when fix in #459 is merged.
jonasbardino added a commit that referenced this pull request Feb 25, 2026
regression test for issue #460 to be enabled when fix in #459 is merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression in the recent addition of transparent compression in userio module (PR345)

3 participants