Fallqs

Math formula and source code

2

# It's a pity that it doesn't look so satisfying when converting Math formulas and source codes via current version. I think it would be useful and convenient if you...

[BUG]: Coati Lora incompatible with Gemini & HybridParallel(pp=1), but runs well with HybridParallel(tp>=2)

1

### 🐛 Describe the bug ## Description I implemented `Coati Lora` before parallel fine-tuning for LlaMA-7B, and found: - `Gemini` runs into _Error(s) in loading state_dict for GeminiCheckpointIO:_ and Train...

[BUG]: Low_Level_Zero plugin crashes with LoRA

14

### Is there an existing issue for this bug? - [X] I have searched the existing issues ### 🐛 Describe the bug The line **808** of `zero/low_level/low_level_optim.py` assumes that every...