Alex Wang

MixtralMlpMixin()这个函数里面moe只是计算专家的logits但是没看到分发逻辑

1

https://github.com/THUDM/SwissArmyTransformer/blob/main/sat/model/official/mixtral_model.py

AutoModel.from_pretrained()里面不能加载hf版本的权重吗

1

File "demo.py", line 44, in model, model_args =AutoModel.from_pretrained('/code/CogVLM-main/vicuna-7b-v1.5', args=argparse.Namespace( File "/opt/conda/lib/python3.8/site-packages/sat/model/base_model.py", line 337, in from_pretrained return cls.from_pretrained_base(name, args=args, home_path=home_path, url=url, prefix=prefix, build_only=build_only, overwrite_args=overwrite_args, **kwargs) File "/opt/conda/lib/python3.8/site-packages/sat/model/base_model.py", line 318, in from_pretrained_base...