DXZDXZ comments

Repositories
Issues
Comments

Results 2 comments of


                                            DXZDXZ

Code release

I also want to know.

[Pytorch] change fused cross entropy backward grad to fp32 and reduce one read/…

@RandMist @yaox12 Hello~ Our experiments found that after applying this change, the output can sometimes become **inf**. This subsequently leads to NaN gradients for the corresponding token during the RMS...