fix: migrate FP8 support from float8_experimental to torchao #618
+5
−5,153
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…date error message for missing dependency
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
PR Summary
Migrates FP8 quantization from deprecated float8_experimental to torchao library in the infinity embedding engine.
torchaodependency inpyproject.tomlshould be moved from dev dependencies to main optional dependencies since it's core FP8 functionality- Version constraint of
torchao>=0.7.0needs stricter upper bound to ensure PyTorch 2.2.1+ compatibility - Missing test coverage for new FP8 quantization implementation
- Documentation updates needed in docs folder as per unchecked items in PR checklist
2 files reviewed, 1 comment
Edit PR Review Bot Settings | Greptile
libs/infinity_emb/pyproject.toml
Outdated
Comment on lines
104
to
105
| [tool.poetry.group.dev.dependencies] | ||
| torchao = "^0.11.0" |
Contributor
There was a problem hiding this comment.
style: Consider moving torchao to main dependencies section with optional=true if it's needed for core FP8 functionality
aryasaatvik
changed the title
aryasaatvik
changed the title
michaelfeil
reviewed
Aug 28, 2025
libs/infinity_emb/poetry.lock
Outdated
| opt-einsum = ["opt-einsum (>=3.3)"] | ||
| optree = ["optree (>=0.13.0)"] | ||
|
|
||
| [[package]] |
Owner
There was a problem hiding this comment.
preferably without adding it to poetry lock
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issue
Fixes FP8 quantization import errors with modern PyTorch versions.
Summary
This PR migrates FP8 quantization support from the deprecated
float8_experimentalpackage to the moderntorchaolibrary. Thefloat8_experimentalpackage has been archived and moved to torchao as of August 2024.Changes:
float8_experimentaltotorchao.quantizationtorchao>=0.7.0as a development dependencyquantize_()API withfloat8_dynamic_activation_float8_weight()Why this is needed:
Checklist
Additional Notes
This change maintains backward compatibility while ensuring FP8 quantization works with modern PyTorch installations.