Can I further fine-tune on the Vicuna-13B base?

[theslugger]

[musabgultekin]

AFAIK, In terms of fine-tuning, there should be no difference from starting from the LLaMA base and vicuna 13b fine tuned models.

[zhisbug]

@theslugger Yes you can. You can either start from the LLama 13b or Vicuna 13B.

There indeed is a difference on which you start fine-tuning. I believe starting from Vicuna at least make sure the model has seen vicuna data before. Closing this ticket. Good luck fine-tuning yourself!

[karan-dalal]

@zhisbug If we start from Vicuna 13B, do we need to modify the format of the dummy.json to have the system prompt?