Lightning-AI
Looking for LLaMA 2?
Meta AI has since released LLaMA 2. Additionally, new Apache 2.0 licensed weights are being released as part of the Open LLaMA project.
To run LLaMA 2 weights, Open LLaMA weights, or Vicuna weights (among other LLaMA-like checkpoints), check out the Lit-GPT repository.
Hi @carmocca, I wonder if it supports Full Parameter Supervised Training for LLaMA2 on customized dataset? Thank you.
Yes, full finetuning is supported via finetune/full.py script given a Llama 2 model provided via the --checkpoint_dir in Lit-GPT.
You can also use a custom dataset given that you prepare it in the right format. You can see the prepare_*.py scripts here for guidance.
Is this repo still intended to be supported given since it seems like the lit-gpt repo supports more and newer models.
