reduce communication to optimize the performance for mixtral tp#4

Open

YuJiankang wants to merge 2 commits intoHabanaAI:mainfrom

YuJiankang:optimize_mixtral_tp

Commits on Apr 2, 2024

deepspeed-fork content for 1.15.1

SW publisher
authored and
Jenkins
committed

Commits on May 10, 2024

reduce communication to optimize the performance for mixtral tp
YuJiankang
committed