[Question] Can OpenChat be instruct-tuned for further downstream tasks?

[Tejaswgupta]

Is it possible to further do instruction tuning on OpenChat with domain specific data? If so, is there any boilerplate that can be used as a starting point. I had earlier fine-tuned LLama-2 on my dataset with trl-sft script, and another try with llama-recipes boilerplate. The time taken by both scripts varied greatly(3x), including the tokenization process and other parameters. The final model however didn't perform well at all with weird and abrupt answers. Therefore, I'm hoping to get some insights if using openchat(or vicuna/wizardlm/llama2-chat) might make a difference?

Thank you for your response.

[imoneoi]

OpenChat takes ShareGPT format data. You can convert the dataset for training using https://github.com/imoneoi/openchat/blob/master/ochat/data/generate_dataset.py

BTW we will update instructions on custom data soon.

[daehuikim]

@imoneoi Any peft would be proper for OpenChat? I tried fine tuning with many fine-tuned models, however i never saw one successfully fine tuned with lora.