Official Implementation of "ToMoE: ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning"
Link to the paper: https://openreview.net/pdf?id=RFHq46pjb6
Note: This repository is actively being updated.
-
Notifications
You must be signed in to change notification settings - Fork 0
[TMLR] Offical Implementation of ToMoE: ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
License
gaosh/ToMoE
About
[TMLR] Offical Implementation of ToMoE: ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
Topics
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published