Liger-Kernel
Liger-Kernel copied to clipboard
linkedin
Efficient Triton Kernels for LLM Training
### π The feature, motivation and pitch ## Summary [Helion](https://github.com/pytorch/helion) is PyTorch's new DSL that compiles to Triton and has powerful autotuning capabilities. It could make Liger kernels easier to...
### π The feature, motivation and pitch OpenAI released GPT-oss: https://github.com/huggingface/transformers/releases/tag/v4.55.0 https://openai.com/index/introducing-gpt-oss/ Would be great to have it supported in liger-kernel. ### Alternatives _No response_ ### Additional context _No response_
## Summary Add GPT-OSS model support, addressing https://github.com/linkedin/Liger-Kernel/issues/848 Completed patching for RoPE, RMSNorm, cross_entropy, and fused_linear_cross_entropy. ### Known Issues - **Gated SwiGLU Patching Support**: The current Hugging Face implementation of...
### π Describe the bug Some of them aren't patched to the correct object. For example: InternVL doesn't have nn.CrossEntropy in its modeling file. The monkey patch should be applied...
## Summary Optimizing Softmax and RMSNorm runtime performance on hidden_size >= 64k ## Testing Done Added large tests for 64K dim - Hardware Type: RTX 3090 - [x] run `make...
I see the function LigerFusedLinearCrossEntropyLoss, I think bias and lin_weight are from the previous layer. If so, maybe `def forward(self, _input, target, lin_weight, bias=None):` is easy to understand. ``` def...
### π The feature, motivation and pitch Can LigerFusedLinearCrossEntropyLoss return per-token logprobs? ### Alternatives _No response_ ### Additional context _No response_
### π The feature, motivation and pitch PyTorch provides native Tensor Parallel techniques for model training. https://docs.pytorch.org/docs/stable/distributed.tensor.parallel.html It would be great to have it supported in liger-kernel. ### Alternatives _No...
### π Describe the bug When trying to finetune `Qwen2.5-1.5B-Instruct` with `DeepSpeed` + LigerKernel, I have encountered the error below: ```bash Traceback (most recent call last): File "/home/user/folder/main.py", line 170,...
### π The feature, motivation and pitch Hi! Thank you for adding support for MTA (https://github.com/linkedin/Liger-Kernel/pull/689) ! Do I understand it correctly, that this implementation only covers post-sm key-query convolution?...
