SwissArmyTransformer icon indicating copy to clipboard operation
SwissArmyTransformer copied to clipboard

Published 20 hours ago • verified publisher icon THUDM

MixtralMlpMixin()这个函数里面moe只是计算专家的logits但是没看到分发逻辑

Open AlenjandroWang opened this issue 1 year ago • 1 comments

https://github.com/THUDM/SwissArmyTransformer/blob/main/sat/model/official/mixtral_model.py

AlenjandroWang avatar Feb 09 '24 13:02 AlenjandroWang