Training new Vicuna based on fully open-source OpenLLaMA

[wilhelmagren]

Hi :wave:

I was wondering if there is any ongoing incentives for training a new Vicuna model based on the fully open source OpenLLaMA? This would ultimately remove the requirement on Vicuna license to be a derivative of the Meta LLaMA non-commercial bespoke license?

Regards,

[ogunoz]

That would be absolutely awesome 😎

[lslslslslslslslslslsls]

That would be great! Vicuna is the only self-hosted model, I've tried, which could execute complex prompts within langchain with very few changes. If we have OpenVicuna, there would be an e2e open source stack, from model to application.

[sarrahbbh]

Hello! isn't this the official public release for Vicuna on HuggingFace?

NOTE: This "delta model" cannot be used directly.
Users have to apply it on top of the original LLaMA weights to get actual Vicuna weights.

Does the statement above mean that I can't just load it from the HF hub and fine-tune it directly? because I tried to and the training loss is constant across all batches (10.375). I thought maybe this is the reason Any help is most appreciated!

[tytung2020]

What do you mean? OpenLlaMA is an open source model trained on the RedPajama dataset. You mean fine tune it using the same method Vicuna was fine tuned from LlaMA?

[lslslslslslslslslslsls]

What do you mean? OpenLlaMA is an open source model trained on the RedPajama dataset. You mean fine tune it using the same method Vicuna was fine tuned from LlaMA?

Yeah, that's it. It would get rid of LLaMA's non commercial license.

[PenutChen]

Is there any plan since OpenLLaMA-7B and OpenLLaMA-13 have been fully trained and released?

[John-Ge]

However, Vicuna does not release the training data.

[jasontian6666]

OpenLlama-based Vicuna would be so awesome to have.