llama2.c
llama2.c copied to clipboard
karpathy
Inference Llama 2 in one file of pure C
Maintainer @karpathy is inactive since many weeks, he made [a single commit few days ago](https://github.com/karpathy/llama2.c/commit/d0237abd32e553317a2bd80ecd5d4c621ddd307a), but it seems more like a "ping signal" than a real commit. We can understand...
As a newcomer to transformers, there are some questions about the matrix multiplication operator in run.c, can the author answer them.  In the forward function, it can be seen...
I found that the dim parameter affects the learning loss and n_layers affects the training speed.   It took 30 minutes. The larger layer only...
I have trained TinyStories to validation loss that Karpathy achieves. I want to fine tune on top of this now. I imagine it is like "resume", but don't reset the...
How to save checkpoints at each step? what is mfu ?
Overriding: compile = False Overriding: eval_iters = 1 Overriding: batch_size = 1 tokens per iteration will be: 1,024 breaks down as: 4 grad accum steps * 1 processes * 1...
One potential optimization is to use a library such as OpenBLAS or Intel's MKL to perform the matrix multiplication in the matmul function. ``` #include void matmul(float* xout, float* x,...
In https://github.com/karpathy/llama2.c/pull/395 support was added to export the model to work with HuggingFace trasformers. However, this only works on the model, and not the tokenizer, so only works with the...
I want to convert this small 1.1B llama2 architecture model [PY007/TinyLlama-1.1B-intermediate-step-240k-503b](https://huggingface.co/PY007/TinyLlama-1.1B-intermediate-step-240k-503b) to llama2.c version. (Layers: 22, Heads: 32, Query Groups: 4, Embedding Size: 2048, Intermediate Size (Swiglu): 5632) Then I...
Is it possible to increase or decrease the size of only some of the layers of the model structure?