PYTHON-5289 Validate ignored bits are 0 on write for bson.BinaryVector#2397
Conversation
…s will sync up with specifications when pull/1812 is merged.
|
Failing tests are unrelated. |
bson/binary.py
Outdated
| unpacked_uint8s = list(struct.unpack_from(format_string, self, position)) | ||
| if padding and n_values and unpacked_uint8s[-1] & (1 << padding) - 1 != 0: | ||
| warnings.warn( | ||
| "Vector has a padding P, but bits in the final byte lower than P are non-zero. In the next major version, they must be zero.", |
There was a problem hiding this comment.
So in the next major version, this warning will become an error to match the behavior in from_vector?
There was a problem hiding this comment.
Yes. We make the significant change (exception not warning) in the next major change and document in changelog and api. The ticket for that is PYTHON-5280 and has 5.0 as its fix version.
There was a problem hiding this comment.
Could you add "pymongo version 5.0" rather than "the next major version". This helps show the user the version (5.0) and that the warning is coming from pymongo.
ShaneHarvey
left a comment
ShaneHarvey
left a comment
There was a problem hiding this comment.
Any docs or changelog we need to update here?
test/test_bson.py
Outdated
| except Exception as exc: | ||
| self.assertIsInstance(exc, ValueError) | ||
| else: | ||
| self.fail("Failed to raise an exception.") |
There was a problem hiding this comment.
This assertion should use assertRaises.
There was a problem hiding this comment.
Yeah. You got me. This looked funny when I reviewed it last week. I'll update to follow best practice.
bson/binary.py
Outdated
| unpacked_uint8s = list(struct.unpack_from(format_string, self, position)) | ||
| if padding and n_values and unpacked_uint8s[-1] & (1 << padding) - 1 != 0: | ||
| warnings.warn( | ||
| "Vector has a padding P, but bits in the final byte lower than P are non-zero. In the next major version, they must be zero.", |
There was a problem hiding this comment.
Could you add "pymongo version 5.0" rather than "the next major version". This helps show the user the version (5.0) and that the warning is coming from pymongo.
|
@ShaneHarvey I've made the changes that you recommended. What do you think? |
ShaneHarvey
left a comment
ShaneHarvey
left a comment
There was a problem hiding this comment.
Suggest adding a versionchanged entry to from_vector docstring.
doc/changelog.rst
Outdated
| when using ``w="majority"``. | ||
| - Ignored bits in a BSON BinaryVector of PACKED_BIT dtype should be set to zero. | ||
| On writes, this is enforced and is a breaking change. | ||
| Reads from the database will not fail, however a warning will be triggered. |
There was a problem hiding this comment.
Suggest referring to BSON encoding vs decoding instead of writes vs reads.
There was a problem hiding this comment.
Ok. Updated with nod to the motivation.
doc/changelog.rst
Outdated
| or the `migration guide <https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/>`_ for more information. | ||
| - Fixed a bug where :class:`pymongo.write_concern.WriteConcern` repr was not eval-able | ||
| when using ``w="majority"``. | ||
| - Ignored bits in a BSON BinaryVector of PACKED_BIT dtype should be set to zero. |
There was a problem hiding this comment.
"Ignored bits" refers to the padding bits right? Should we say "padding" to avoid introducing a new term?
There was a problem hiding this comment.
Will update to clarify.
doc/changelog.rst
Outdated
| when using ``w="majority"``. | ||
| - When padding is set, ignored bits in a BSON BinaryVector of PACKED_BIT dtype should be set to zero. | ||
| When encoding, this is enforced and is a breaking change. | ||
| It is not enforced when decoding, so reading from the database will not fail, however a warning will be triggered. |
There was a problem hiding this comment.
Should this state that in PyMongo >= 5.0, it will be enforced?
| Binary.from_vector([255], BinaryVectorDtype.PACKED_BIT, padding=7) | ||
|
|
||
| # Test form of Binary.from_vector(BinaryVector) | ||
| assert padded_vec == Binary.from_vector( |
There was a problem hiding this comment.
Should we test that the warning is raised as well?
There was a problem hiding this comment.
Definitely. Thanks! Added.
…ne on encoding AND decoding.
3 participants
This is the implementation of the spec changes applied in specifications/pull/1812.