grad norm keeps being nan

[JMoonr]

Dear author, thank you for your work and sharing the code, I try to train cape_VoV99_1600x640_24ep_wocbgs_load_dd3d_pretrain following your instructions, but the log shows that grad_norm keeps being nan, do you know why this happens and how to fix this?

[kaixinbear]

Dear JMoonr, while conducting the experiment, I encountered the same issue; however, after careful checking, I determined that it would not significantly impact the final results. As a result, I chose to disregard this phenomenon. Please tell me if you know how to solve this problem.