Open
Conversation
Author
|
I am running into one issue though. When I try to create the full domain for I get this error: But I don't think I should be? |
Member
|
I'll have to look closer later, it fails even with |
Author
|
This pull request has been linked to: |
added 2 commits
August 24, 2022 09:45
* Previously, the `tile_max` was only correctly calculated for `np.int64`. This correctly set the `dim_max` to `dtype_max - tile` but only for `np.int64` * All other integer dtypes had `dim_max` set to `dtype_max` which resulted in a domain range that was too large * We also need to account for when the tile extent is larger than the range of the full domain. For an instance, when we have a dim dtype of `np.int8`, the default tile extent of 10000 will be much larger than the range of the full domain
38634c1 to
def094a
Compare
* I was mistakenly under the impression that `tiledb.Dim(domain=(-128, 126), tile=254, dtype=np.int8)` was a valid tile extent for int8. But since the max value of int8 is 128, not 256, we can't have an extent larger than that.
Author
|
I was mistakenly under the impression that |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tile_maxwas only correctly calculated fornp.int64. This correctly set thedim_maxtodtype_max - tilebut only for
np.int64dim_maxset todtype_maxwhichresulted in a domain range that was too large
the range of the full domain. For an instance, when we have a dim
dtype of
np.int8, the default tile extent of 10000 will be muchlarger than the range of the full domain