Lightning-AI
Add Salamandra models - Multilingual LLaMA-based models including minority languages
Hi,
I would like to propose adding the Salamandra models to lit-gpt. Salamandra is a model based on LLaMA that has been trained from scratch using 35 European languages, including minority languages like Catalan and Basque, making it unique compared to other models primarily focused on English.
Salamandra comes in four versions:
The Instruct versions have been fine-tuned for instruction-based tasks, while the non-instruct versions remain general-purpose.
Adding Salamandra to lit-gpt would enrich the framework by introducing multilingual capabilities, including support for less-represented languages. This would allow developers and researchers to work with languages beyond English, making lit-gpt more versatile and inclusive.
I believe this addition could make lit-gpt a more comprehensive framework for multilingual tasks and help it stand out in non-English projects.
Looking forward to your feedback!
I would be open to adding these models. If it helps, I've recently written a how-to guide here: https://github.com/Lightning-AI/litgpt/blob/main/tutorials/developer-docs/adding-models.md
