lightning-thunder
lightning-thunder copied to clipboard
Notebook: Make llm.c thunder-compatible and use native llm.c CUDA kernels
As per title!
We kick-off with a PyTorch reference model, then replace some of its parts with the llm.c's CUDA kernels. We also test the outcome against the C implementation (using provided tools) and identify potential future work.
Maybe we want to actually provide C-Python bindings for tightener benchmarks?
Check out this pull request on ![]()
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
@nikitaved should we merge this one so we keep it as a reference? Maybe let's add a WIP caveat, demonstration purposes only.
tagging older Draft PR's at later for tidiness
@nikitaved feel free to keep chipping at this or reopen if this gets back being active