llm.c
llm.c copied to clipboard
karpathy
Why not Mojo?
Serious question here. If you're going down to the metal, Mojo provides a potentially large speedup and the language would significant benefit from this work. Either way - love the work. Thank you!
C is just the simplest thing super happy to link to any mojo port (or anything else) from the main readme! we can benchmark and compare them all :)
Make a fork. Happy to help.
https://github.com/tairov/llama2.mojo
The way to do it would be to add to the Makefile like https://github.com/ggerganov/llama.cpp/blob/master/Makefile
Then as part of the build you pass LLM_USEMOJO or whatever and it would compile the mojo specialized functions.
I would start with linking a MOJO version of matmul_backward() in train_gpt2.c .
Minimal fork just to show the Makefile. I'll work on porting matmul_backward() and linking it.
https://github.com/chadbrewbaker/llm.c/tree/mojo
@chadbrewbaker does CFLAGS = -O3 -Ofast works togheter on your machine? i got this problem 08e618e73f13544f24122196fa1890a39868f94f, and needed to use -O3 alone, or disable the -ffast-math flag which was called by -Ofast, it messes up the floating point arithmetics
Lol. "Fast math" does crazy rounding modes - best to avoid unless you know what you are doing. Just did a deep dive and Modular/Mojo still don't have their C libraries available on OSX and OSX static linking isn't officially supported yet.
https://docs.modular.com/engine/reference/cli/
Best for now is to write a train_gpt2.mojo that links each train_gpt2.c functions. It is inside out but will at least work for benchmarking. Might need this so main is removed from the object code.
#ifndef LLM_NO_MAIN
int main() {
printf("Main program is compiled.\n");
return 0;
}
#endif
There. I changed main() to be removable. Now mojo should be able to link in any of the functions for benchmarking.
https://github.com/chadbrewbaker/llm.c/blob/56b41fe4bad74db1075c395149bdc9eeb54d0911/train_gpt2.c#L1117
If I am not mistaken, Mojo core is not even an open lang.
Not completely yet, right. It will open sourced successively. This includes the compiler stack and parts of MAX engine. The chosen license is Apache 2.0 with LLVM-Exceptions.
I just uploaded a Mojo port to github. https://github.com/dorjeduck/llm.mojo Not to be taken too serious yet, an early version of someone just learning Mojo ....
I just uploaded a Mojo port to github. https://github.com/dorjeduck/llm.mojo Not to be taken too serious yet, an early version of someone just learning Mojo ....
LGTM. You want to submit a PR? I would make a llm.c/dev/mojo directory to put it in?
Threw up what @dorjeduck had to a PR. https://github.com/karpathy/llm.c/pull/119
Rather than pollute the root Makefile, should I add a llm.c/dev/mojo/Makefile ?
Most simple benchmark is to run it with /usr/bin/time (-v Linux, -l Darwin) to get operating system statistics?
dont open.a PR with my source code yet, its not ready. I will do when i feel comfortable with it