Liger-Kernel icon indicating copy to clipboard operation
Liger-Kernel copied to clipboard

Published 20 hours ago • verified publisher icon linkedin

GeGLU kernel numerical issue

Open Tcc0403 opened this issue 1 week ago • 0 comments

🐛 Describe the bug

The current Liger geglu kernel is not close to torch's implementation, even in fp32. test_geglu.py can't pass with lower tolerances.

Some models couldn't pass the convergence test with low tolerance due to this issue. We need to explore whether there is a implementation that is closer to torch's.

Reproduce

No response

Versions

Liger Kernel version: 0.6.4 PyTorch version: 2.7.1+cu126

Tcc0403 avatar Nov 28 '25 20:11 Tcc0403