llm-foundry
llm-foundry copied to clipboard
mosaicml
MPT-7B Finetuning Jupyter notebook request
@vchiley @samhavens @alextrott16 , i was going through the MPT-7B model fine tuning documentation. It is def well written but quite hard to grasp in the first look.
Therefore, I am putting forth this request to create a fine-tuning jupyter notebook, that folks can use to train using their local GPU or cloud GPUs (paid like A100s etc)
It would be great to have the same as a jupyter notebook. From loading the mpt model , to loading the instruction set or combination of instruction sets and then finally running the model fine-tuning.
@bmosaicml @coryMosaicML @nik-mosaic @abhi-mosaic hope you could have a look into this
@GeorvityLabs I'm putting together a much more detailed example to illustrate the finetuning workflow. The current plan does not include a notebook, though.
We might opt to set up example notebooks in the future, but I think that will have to be a later decision. We'll certainly consider this request when deciding how to curate examples.
@alextrott16 hope you guys can make a blog post on how to fine-tune MPT-7B on alpaca , dolly , oasst kinda datasets, would be of great help!
@vchiley @samhavens @alextrott16 , i was going through the MPT-7B model fine tuning documentation. It is def well written but quite hard to grasp in the first look.
Therefore, I am putting forth this request to create a fine-tuning jupyter notebook, that folks can use to train using their local GPU or cloud GPUs (paid like A100s etc)
It would be great to have the same as a jupyter notebook. From loading the mpt model , to loading the instruction set or combination of instruction sets and then finally running the model fine-tuning.
One can try-out Q Blocks GPUs if one does not have a local setup or Colab Pro
I am going to close this issue for now, but the request for a notebook has been noted. We've added a more concrete, runnable finetuning example that partially addresses this.
@alextrott16 yup, a blog post regarding instruction fine-tuning would be great.