refactor: introduce StorageMapKey and HashedStorageMapKey Word wrappers #2431
refactor: introduce StorageMapKey and HashedStorageMapKey Word wrappers #2431Farukest wants to merge 6 commits into0xMiden:nextfrom
StorageMapKey and HashedStorageMapKey Word wrappers #2431Conversation
6082b2c to
435b677
Compare
| /// underlying SMT. Wrapping the hashed key in a distinct type prevents accidentally using a raw | ||
| /// key where a hashed key is expected and vice-versa. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, WordWrapper)] | ||
| pub struct HashedStorageMapKey(Word); |
There was a problem hiding this comment.
Despite my initial suggestion, I'd now name this to StorageMapKeyHash, since the naming pattern we usually have is to append things at the end, e.g. StorageSlot and StorageSlotContent, StorageSlotName, etc.
| /// Returns the leaf index in the SMT for this raw key. | ||
| /// | ||
| /// This first hashes the key, then converts it to a leaf index. | ||
| pub fn to_leaf_index(&self) -> LeafIndex<SMT_DEPTH> { | ||
| self.hash().to_leaf_index() | ||
| } |
There was a problem hiding this comment.
I'd remove this implementation to make it clearer that only a StorageMapKeyHash can be converted into a LeafIndex - i.e. not StorageMapKey but StorageMapKeyHash are used as the key in an SMT.
| @@ -140,8 +144,8 @@ impl StorageMap { | |||
| /// | |||
| /// Conceptually, an opening is a Merkle path to the leaf, as well as the leaf itself. | |||
| pub fn open(&self, raw_key: &Word) -> StorageMapWitness { | |||
There was a problem hiding this comment.
| pub fn open(&self, raw_key: &Word) -> StorageMapWitness { | |
| pub fn open(&self, key: impl Into<StorageMapKeyHash>) -> StorageMapWitness { |
I think we should update all the StorageMap APIs that take a key to take it in this form. So, any type that can be converted into a StorageMapKeyHash. This should provide good type safety: If you provide a StorageMapKey, it will be converted (and hashed) into a StorageMapKeyHash while you can also provide a StorageMapKeyHash if that's all you have. This way we allow using both types of keys while users don't have to think about those details.
Maybe this will also require refactoring StorageMap, PartialStorageMap and StorageMapWitness to use BTreeMap<StorageMapKey, Word> internally, but take a look.
| @@ -98,7 +98,7 @@ impl PartialStorageMap { | |||
| /// - [`Word::empty`] if the key is tracked by this map and does not exist, | |||
| /// - `None` if the key is not tracked by this map. | |||
| pub fn get(&self, raw_key: &Word) -> Option<Word> { | |||
There was a problem hiding this comment.
I would do the same impl Into<StorageMapKeyHash> change for PartialStorageMap APIs.
|
All done @PhilippGackstatter 👍 |
35e637f to
e44465f
Compare
PhilippGackstatter
left a comment
There was a problem hiding this comment.
Looks good!
I left a few more comments, after which I think we can probably merge.
I think there are more places that we should eventually refactor, but it will be better to do this in a follow-up PR to keep this PR more reasonable in scope.
Follow-ups
StorageMap::with_entries might also benefit from changing to (StorageMapKey, Word) constructor for type safety, e.g. the API is clearer on which of the two words is the key and the value. Also, it will avoid any doubt about whether the provided key needs to be the raw key or the hashed key.
We should also make use of this type in these places which will make the data store api much clearer:
| .iter() | ||
| .find(|(k, _)| StorageMapKey::from_raw(**k).hash() == key_hash) | ||
| .map(|(k, v)| (*k, *v)) | ||
| .unwrap_or_default(); |
There was a problem hiding this comment.
I think we should be able to keep self.entries.get here?
There was a problem hiding this comment.
I'm not sure that we can use get here anymore, since we don't have the original raw key here, only its hash, while the entries map uses raw keys.
But I'm wondering whether we need to have key: impl Into<StorageMapKeyHash> here in the first place: I agree with you that it will be much more convenient from the user standpoint, but will the hashed key be ever used by... well, user? It seems like this is an internal value only used to balance the tree, but will there be any cases when user will operate with hashed keys directly? If not, maybe it makes sense to keep the raw key (StorageMapKey in our case) as a parameter of this procedure? Since it will allow us just to get the value using self.entries.get method, saving us the burden of getting the raw key.
There was a problem hiding this comment.
Ah, true, thank you! We actually need to pass the raw key to StorageMapWitness, and so if we only have the hash, this won't work. So indeed we should take key: &StorageMapKey as an input to open. Technically for get we could use StorageMapKeyHash without problems, but it may be better to require StorageMapKey for API uniformity. My bad for suggesting StorageMapKeyHash!
| /// Accepts either a [`StorageMapKey`] (which will be hashed) or a [`StorageMapKeyHash`] | ||
| /// directly. | ||
| pub fn get(&self, key: impl Into<StorageMapKeyHash>) -> Option<Word> { | ||
| let hash_word: Word = key.into().into(); |
There was a problem hiding this comment.
| let hash_word: Word = key.into().into(); | |
| let hash_word = key.into().as_word(); |
Nit: This might be a bit easier to read.
| impl From<Word> for StorageMapKey { | ||
| fn from(word: Word) -> Self { | ||
| Self(word) | ||
| } | ||
| } |
There was a problem hiding this comment.
This is technically correct, but we previously decided not to implement From<Word> for these kinds of word-wrapper types, so that the "provenance" of the StorageMapKey becomes clearer. E.g. not every Word is semantically a StorageMapKey that is valid for a given map, and you need to use the deliberately less convenient from_raw API to vouch that the key is valid.
|
|
||
| for raw_key in raw_keys.into_iter() { | ||
| let hashed_map_key = StorageMap::hash_key(raw_key); | ||
| let hashed_map_key: Word = StorageMapKey::from_raw(raw_key).hash().into(); |
There was a problem hiding this comment.
I would also change the constructors of StorageMapWitness to use StorageMapKey, e.g.:
StorageMapWitness::newStorageMapWitness::new_unchecked
So that we don't need to handle raw_key here. We can also rename raw_key to use key: StorageMapKey, since it's clear from the type now that it's the "raw key".
Fumuran
left a comment
There was a problem hiding this comment.
Looks good, thank you! In addition to Philipp's comments, I left just one small optional nit inline.
| let hash_word: Word = key_hash.into(); | ||
| let smt_proof = self.smt.open(&hash_word); |
There was a problem hiding this comment.
optional nit: we can optimize this a bit
| let hash_word: Word = key_hash.into(); | |
| let smt_proof = self.smt.open(&hash_word); | |
| let smt_proof = self.smt.open(&key_hash.into()); |
1c9ce8c to
3ac810b
Compare
|
All done @PhilippGackstatter @Fumuran let me know if I missed something :) |
6bb353f to
5d7b47c
Compare
5d7b47c to
e3fb5c2
Compare
Summary
Closes #1864
Introduces type-safe
Wordwrappers for storage map keys to prevent accidentally mixing up raw and hashed keys:StorageMapKey(Word)— wraps a raw, user-chosen key withhash() -> HashedStorageMapKeyandto_leaf_index()methods.HashedStorageMapKey(Word)— wraps a hashed key withto_leaf_index()method.Both types use
#[derive(WordWrapper)]frommiden-protocol-macros.The utility functions
StorageMap::hash_key,StorageMap::map_key_to_leaf_indexandStorageMap::hashed_map_key_to_leaf_indexare removed and replaced by methods on the new types. All internal and external call sites are updated accordingly.