SuperRobin

Results 49 comments of SuperRobin

How to implement in vision transformer?

any updates?

Any plan to share the pre-trained models?

+1

Any plans to release a sample code for MINI-LM distillation?

+1

多机多卡训练容易超时，超时的话如何自动从已经保存的模型恢复训练？

any update?

[BUG]: ZERO DDP error: the synchronization of gradients doesn't exit properly when train Mixtral-8x7B-v0.1 with GeminiPlugin or HybridParallelPlugin

any update for this?

[BUG]: ZERO DDP error: the synchronization of gradients doesn't exit properly when train Mixtral-8x7B-v0.1 with GeminiPlugin or HybridParallelPlugin

the same problem

[moe] support mixtral

Use the latest commit code, however after several iterations，the loss becomes NaN

[moe] support mixtral

Just leave it?

[moe] support mixtral

Is there a approximate release time？ Since the latest commit still contains bugs.

[lora] support lora for Gemini

any update of this work?