Finetuning in unsupported language

[jumasheff]

My language was not on the list of 20 languages the original model was trained on. Is it possible to finetune llama with a dataset in a language that was not included in the base model?

[hmrc87]

I tried finetuning GPT2 which apparently was only trained with english language with german input text and the result is bad. I guess it takes a whole lot of more training in order to generate a new language.

In short: I think you probably need another model

[lxe]

You can absolutely do this. Tell me how it goes!