[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… metadata ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16403
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 1 Unrelated FailureAs of commit 8aeb5f4 with merge base 9a30bd3 ( NEW FAILURES - The following jobs have failed:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
SS-JIA
pushed a commit
that referenced
this pull request
Dec 27, 2025
… metadata ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) ghstack-source-id: 331218513 Pull Request resolved: #16403
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Context
With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by
vTensorused to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate theGPUMemoryLayoutwhich would be needed to compute storage descriptors such as image extents.This diff addresses the problem by introducing the
PackedDimInfostruct tovTensor, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents.PackedDimInfoIntroduced the
PackedDimInfostruct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit.The
PackedDimInfostruct contains:packed_dim: Which dimension is tightly packed (WHCN index) / contiguous in memorypacked_dim_padded: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/storesouter_packed_dim: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal topacked_dimouter_packed_dim_padded: Whether outer packed dim is padded (tiled only)Changes
padded_sizes_member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of usingsizes_directlyDifferential Revision: D89832382