matrixssy

> The same question. I am working for it, but I am not sure if it will be accepted.

see https://github.com/NVIDIA/Megatron-LM/pull/667

> Hi, Please refer to this [script](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core/transformer/moe#a-detailed-moe-script) for MoE/Mixtral training. Great! But I would like to know how to convert Hugging Face (HF) weights to Megatron (MG) format, and if...

Me too, Have you solved it yet?

> > Me too, Have you solved it yet? > > I just down the version down the version can avoid the error, but I get bad images or the...