Seu_ZL

Same Problem

have you solve this problem bro?I face the same problem o(╥﹏╥)o

> 抱歉，我犯了个错误。 这个[问题](https://github.com/deepspeedai/DeepSpeed/issues/6767#issuecomment-2489408575)已经指出，梯度裁剪发生在 deepspeed 的optimizer.step()中。 Hello, I have also reviewed this issue and I am a bit lacking in knowledge about these very low-level things. However, based on my training...

> > 好的，经过一番调查后我发现，至少在我的情况下，梯度剪辑正在发生，但变形金刚训练器记录了错误的内容。 > > 在 Transformer 的训练器中，通过调用以下命令记录梯度范数`get_global_grad_norm()`： > > ``` > > if (is_accelerate_available() > > and self.accelerator.distributed_type == DistributedType.DEEPSPEED): > > grad_norm = model.get_global_grad_norm() > > # In...