Liger-Kernel icon indicating copy to clipboard operation
Liger-Kernel copied to clipboard

Published 20 hours ago • verified publisher icon linkedin

[WIP] Update benchmark data

Open Tcc0403 opened this issue 7 months ago • 6 comments
trafficstars

Summary

Rerun all benchmarks scripts to get the latest data, so we can have a reliable baseline for future optimization.

Note: orpo failing with compile=True (plotting with old data for now), qwen2vl_mrope script failed.

A complete comparison figure will be uploaded in this PR later.

Fused Linear Chunked Loss

Alignment

  • [x] CPO

    speedfused_linear_cpo_loss_speed

  • [x] DPO

    speeddpo_loss_speed

  • [x] KTO

    speedkto_loss_speed

  • [x] ORPO

    speedfused_linear_orpo_loss_speed

  • [x] SimPO

    speedfused_linear_simpo_loss_speed

Distillation

  • [x] JSD
    speeddistill_jsd_loss_speed

Others

  • [x] Cross Entropy

    speedcross_entropy_speed

  • [x] Fused Linear Cross Entropy

    speedfused_linear_cross_entropy_speed

  • [x] JSD

    speedjsd_speed

  • [ ] Fused Linear JSD

    speed

  • [x] DyT

    speeddyt_speed

  • [x] Embedding

    speedembedding_speed

  • [x] GeGLU

    speedgeglu_speed

  • [x] GroupNorm

    speedgroup_norm_speed

  • [x] KL Div

    speedkl_div_speed

  • [x] LayerNorm

    speedlayer_norm_speed

  • [x] RMSNorm

    speedrms_norm_speed

  • [x] RoPE

    speedrope_speed

  • [ ] Swiglu

    speed

  • [x] TVD

    speedtvd_speed

Testing Done

  • Hardware Type: <BLANK>
  • [ ] run make test to ensure correctness
  • [ ] run make checkstyle to ensure code style
  • [ ] run make test-convergence to ensure convergence

Tcc0403 avatar Apr 02 '25 10:04 Tcc0403