Pingzhi Li comments

Repositories
Issues
Comments

Results 4 comments of


                                            Pingzhi Li

Finetuning / Distillation for Mixtral?

Thank you for your interest! This is an important suggestion - to evaluate the final performance of MC-SMoE on Mixtral. We haven't tried it yet, as distillation on large models...

NotImplementedError: This model does not have a tensor parallel plan

Hi thank you for your interest. I think this is due to version issue. Could you try downgrading `transformers` to 4.34.1?

[Bug] Stuck at NCCL initialization when TP>1

solved by adding `NCCL_P2P_DISABLE=1`, but still confused and worried about the performance. would greatly appreciate it if someone could kindly help on this. :)

Is the data replication script code library for Table 2 available?

Hi thanks for your question. During both merging and compression, the scripts will by default run evaluation every N steps. You should already have the evaluation results now.