2024 Q4 Roadmap

[ByronHsu]

trafficstars

As the community grows, keeping track of issues and PRs becomes more and more challenging. This pinned issue will serve as the central place to manage the progress in 2024 Q4 (~2024/12). Here we only list the important and top level issues/PRs.

Progress Tracker

Format: [title] [link], [contributor] / [reviewer (maintainer)]

New Features: Liger to Post Training

https://github.com/linkedin/Liger-Kernel/issues/371

Models

• llama 3.2 vision (https://github.com/linkedin/Liger-Kernel/pull/282), @tyler-romero / @shivam15s
• pixtral 12B (https://github.com/linkedin/Liger-Kernel/pull/253), @AndreSlavescu / @ByronHsu
• Flux model (https://github.com/linkedin/Liger-Kernel/issues/73), call for help / @qingquansong
• DeepseekV2 (https://github.com/linkedin/Liger-Kernel/issues/129), call for help / @qingquansong
• Gemma2 (https://github.com/linkedin/Liger-Kernel/issues/127), call for help / @yundai424
• Qwen2-VL (https://github.com/linkedin/Liger-Kernel/pull/283), @tyler-romero / @shivam15s
• Jamba (https://github.com/linkedin/Liger-Kernel/pull/214), @yubofredwang / @ByronHsu

Kernels

• TVD loss, https://github.com/linkedin/Liger-Kernel/pull/324 / @qingquansong @lancerts
• JSD loss, https://github.com/linkedin/Liger-Kernel/pull/300 / @qingquansong @lancerts
• GroupNorm (https://github.com/linkedin/Liger-Kernel/pull/225), @denti / @shivam15s
• Z Loss in cross entropy (https://github.com/linkedin/Liger-Kernel/pull/239), @Tcc0403 / @shivam15s
• Flash Attention in Triton (https://github.com/linkedin/Liger-Kernel/pull/275), @remi-or / @shivam15s
• Conv2d (https://github.com/linkedin/Liger-Kernel/pull/228), @AndreSlavescu / @lancerts
• Triton mm int8 x int2(https://github.com/linkedin/Liger-Kernel/pull/195), @mekkcyber / @ByronHsu

Testing

• assert_verbose_allclose (https://github.com/linkedin/Liger-Kernel/pull/261), @Tcc0403 / @ByronHsu

Patching

• Weights are not copied for instance patching (https://github.com/linkedin/Liger-Kernel/pull/280), @shimizust / @ByronHsu

Community Sync

TBD