lightning-thunder icon indicating copy to clipboard operation
lightning-thunder copied to clipboard

Automated litgpt benchmarks

Open KaelanDt opened this issue 7 months ago • 0 comments

🚀 Feature

As suggested by @Borda it would be nice to have automated litgpt benchmarks for thunder, in the same format as the HF model benchmarked through https://github.com/Lightning-AI/lightning-thunder/blob/main/.github/run-benchmark-as-lit-jobs.py.

Motivation

More coverage from automated benchmarks, giving a better overview of thunder's performance.

cc @borda

KaelanDt avatar May 16 '25 14:05 KaelanDt