Neo Zhang

Results 3 comments of Neo Zhang

Getting an Out of Memory (OOM) error when training a 7B model on 4x A100 80G GPUs using GRPO.

可以尝试将使用KL散度设置为False，我试过能大量降低System Memory Utilization `actor_rollout_ref.actor.use_kl_loss=False `

请问grpo的lora训练完成后，保存在actor路径下的.pt文件是已经合并lora之后的模型参数吗？

想请问一下，可以将被训练的基座模型与global_step_xx/actor/lora_adapter下的LoRA直接合并吗？没有必要将.pt文件转为hf格式之后再合并lora权重吧

[algo] feat: support router replay

I would like to inquire about the latest progress of this project. Does R3 support the training of Megatron+vLLM? Does Megatron need to use the PR version you submitted: https://github.com/NVIDIA/Megatron-LM/pull/2101/files?