Llama 3.2 training scripts?

[victorb]

System Info

N/A

Information

• [X] The official example scripts
• [ ] My own modified scripts

🐛 Describe the bug

The llama.com website states:

The open-source AI model

So I'm looking for the training scripts (the source part of a model release) of the Llama 3.2 model that Facebook/Meta used for training the released model, which I suppose would be available in this repository but I cannot find it.

Error logs

N/A

Expected behavior

I'd expect an open source model to have the source available to train my own version of the model.

[init27]

Hey @victorb if you mean you're looking for a script to help you get started with Fine-Tuning, we have a reference here.

However, at the moment-we haven't open sourced the pre-training scripts

Hope this helps!

[victorb]

However, at the moment-we haven't open sourced the pre-training scripts

The weights isn't "source code" nor "open" (seems I need to sign an agreement to download, and models has usage restrictions), and seems the training scripts/data isn't "open" either.

So what exactly is "open source" about Llama?

Sounds like either there is sources missing here, or you need to update the website to remove the "open source" part.

[victorb]

@init27 as far as I can tell, the website still falsely claims "The open-source AI model" and "how Llama is the leading open source model family" while no training scripts nor the data source (beyond "mix of public and private data) is provided.

How is this considered "completed" all of a sudden? Were the training scripts and training data disclosed somewhere?

[umanniyaz]

Do we have any training notebooks for Domain Adaptive Pre training or Continual training for text generations.Not fine tuning everywhere there is instruction fine tuning not Continual training @init27 Please share such notebook script