How to finetune mistral-7b with fastchat?

[Jerry-hyl]

When I use fastchat to finetune llama2, everything is ok. But when I want to finetune mistral, it shows that "transformer layer not found". I know the main reason is that the value of --fsdp_transformer_layer_cls_to_wrap should not be LlamaDecoderLayer. But what should it be set? Hope you can solve my problem. Many thanks!

[zouyingcao]

same question, please! I tried to change the LlamaDecoderLayer to MistralDecoderLayer, but there are too many warnings or performance drops quickly after sft. Hope for solution.