can the llm2vec model be reused for chat (post LLM2vec fine-tune for SFT) ?

[StephennFernandes]

Hi there, I am building multilingual chat LLMs often in Encoder, EmcDec models like XLM-R, mT6 and parallel bilingual corpus is used for model to learn cross lingual references and generalizes on multilingual dataset by cross lingual generalization.

given that CLM is really bad for such tasks, LLM2vec seems like there is still hope for training Decoder only LLMs on such corsa lingual generalizations.

I just wanted to know if I could post training multilingual LLMs on LLM2vec style training could I use the same LLM for SFT training later on ?

[louieworth]

I have the same concern,.

[vaibhavad]

I did not fully understand your question. All LLM2Vec models and code is public and can be used for further fine-tuning on any dataset/task.

We have not trained LLM2Vec on any cross lingual tasks, but you can definitely go ahead and do it.

Let me know if you have any more concerns or questions

[vaibhavad]

Feel free to re-open if you have any more questions regarding this issue.