lightning-thunder icon indicating copy to clipboard operation
lightning-thunder copied to clipboard

Notebook: Make llm.c thunder-compatible and use native llm.c CUDA kernels

Open nikitaved opened this issue 1 year ago • 1 comments

As per title!

We kick-off with a PyTorch reference model, then replace some of its parts with the llm.c's CUDA kernels. We also test the outcome against the C implementation (using provided tools) and identify potential future work.

Maybe we want to actually provide C-Python bindings for tightener benchmarks?

nikitaved avatar Apr 18 '24 16:04 nikitaved

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@nikitaved should we merge this one so we keep it as a reference? Maybe let's add a WIP caveat, demonstration purposes only.

lantiga avatar Jul 01 '24 08:07 lantiga

tagging older Draft PR's at later for tidiness

@nikitaved feel free to keep chipping at this or reopen if this gets back being active

lantiga avatar Jul 03 '24 19:07 lantiga