C080

Hi! Try loading gpt-XL weights and fine tune to your prompt-answer dataset, It should be able to produce your desired output

It could pick up Turkish if it has been trained on a multi-lingual dataset with turkish inside! Anyway try using two layers of Google Translates after & before so all...

same issue I manage to run the pretraining script by setting model.mm_cfg.llm.from_pretrained=null and it works but is he seems to pretraing the llm from scratch (?)