DPO 损失函数计算NaN问题查找

[hexiao0275]

{"loss": 0.86816406, "grad_norm": 1.0, "learning_rate": 0.0001, "memory(GiB)": 16.94, "train_speed(iter/s)": 0.07281, "rewards/chosen": 0.31445312, "rewards/rejected": -1.109375, "rewards/accuracies": 0.5, "rewards/margins": 1.421875, "logps/chosen": -8.75, "logps/rejected": -169.0, "logits/chosen": -1.1953125, "logits/rejected": -0.734375, "nll_loss": 0.37695312, "epoch": 0.0008, "global_step/max_steps": "1/3750", "percentage": "0.03%", "elapsed_time": "3s", "remaining_time": "4h 1m 38s"} {"loss": 0.0, "grad_norm": 1.0, "learning_rate": 0.0001, "memory(GiB)": 29.28, "train_speed(iter/s)": 0.198544, "rewards/chosen": NaN, "rewards/rejected": NaN, "rewards/accuracies": 0.0, "rewards/margins": NaN, "logps/chosen": NaN, "logps/rejected": NaN, "logits/chosen": NaN, "logits/rejected": NaN, "nll_loss": NaN, "epoch": 0.004, "global_step/max_steps": "5/3750", "percentage": "0.13%", "elapsed_time": "15s", "remaining_time": "3h 11m 11s"} {"loss": 0.0, "grad_norm": 1.0, "learning_rate": 0.0001, "memory(GiB)": 36.7, "train_speed(iter/s)": 0.255158, "rewards/chosen": NaN, "rewards/rejected": NaN, "rewards/accuracies": 0.0, "rewards/margins": NaN, "logps/chosen": NaN, "logps/rejected": NaN, "logits/chosen": NaN, "logits/rejected": NaN, "nll_loss": NaN, "epoch": 0.008, "global_step/max_steps": "10/3750", "percentage": "0.27%", "elapsed_time": "29s", "remaining_time": "3h 2m 46s"}

loss一直是NaN的状态 ，怎么查找是哪条训练数据的问题呢

[Jintao-Huang]

一般 batch_size大些不容易nan

[github-actions[bot]]

This issue has been inactive for over 3 months and will be automatically closed in 7 days. If this issue is still relevant, please reply to this message.