Using llama3 through lit lama

[fireyanci]

If I want to use llama3 through lit lama, how can I modify it? I found that the model structure of llama3 has changed

[rasbt]

While this repository is only focused on the first Llama model to keep the code as simple and readable as possible, we have the LitGPT repository (which is an extension of Lit-Llama) that also now supports Llama 3 as of Friday, thanks to @awaelchli

Hopefully this addressed your needs. And please let us know in case you bump into any issues.

[fireyanci]

While this repository is only focused on the first Llama model to keep the code as simple and readable as possible, we have the LitGPT repository (which is an extension of Lit-Llama) that also now supports Llama 3 as of Friday, thanks to @awaelchli

Hopefully this addressed your needs. And please let us know in case you bump into any issues.

Thank you for your answer

[fireyanci]

While this repository is only focused on the first Llama model to keep the code as simple and readable as possible, we have the LitGPT repository (which is an extension of Lit-Llama) that also now supports Llama 3 as of Friday, thanks to @awaelchli

Hopefully this addressed your needs. And please let us know in case you bump into any issues.

Due to meta-ai lama-3-8b not using sentencePiece, but litgpt/tokenizer. py still uses sentencePiece. When I use the litgpt chat -- checkpoint_dir E: \ checkpoint \ meta llama \ Meta-Llama-3-8B Instrument RuntimeError: Internal: could not part ModelProto from E: \ checkpoint \ meta llama \ Meta Llama-3-8B Instrument \ tokenizer. model Is there a new version of tokenizer. py that is compatible with llama-3 in Litgpt

[rasbt]

I just saw your comment also in https://github.com/Lightning-AI/litgpt/issues/1333. Let's continue the discussion there.

[awaelchli]

@fireyanci A few notes here:

• LitLlama is no longer being developed. The replacement is LitGPT. If you want to use Lama3 without major effort, please use LitGPT directly.
• It is possible to make changes in LitLlama to support Llama3, but you'll have to update several files. If this is very important for you, I suggest start by copying over model.py, config.py and tokenizer.py and then work from there. But again, at this point you could use LitGPT directly.
• In LitGPT, the tokenizer supports loading both sentenciepiece and huggingface tokenizers (including LLama3). Follow the download tutorial to get the LLama3 weights from HF.
• Your messages indicate you are using a model called "Meta-Llama-3-8B Instrument". This does not exist. meta-llama/Meta-Llama-3-8B-Instruct is the correct name. Please try with this again.