Daya Guo
Daya Guo
Set share=True in app.py and use public URL.
Firstly, alpaca data is not intended for conversation but rather for instructional purposes. You should consider leveraging other data specifically designed for conversation. It appears that Bloomz-7b1-mt is a model...
I see. Please adjust batch size or epochs [here](https://github.com/project-baize/baize-chatbot/blob/c02f40b664602f77d676e30d6f1b72a2f3ce205a/finetune.py#L22-L25), because you only update 50k/64= 781 steps. However, we update about 3000 steps using all data.
You need to install transformer `pip install git+https://github.com/huggingface/transformers.git`, since the latest transformers doesn't support LLaMa yet.
When you load 7b, do you also using 8 Bit? or only use fp16?
I attempt to load 13b with 8 bits and it works without issue. It appears that the error was caused by bitsandbytes. Unfortunately, I am unsure of how to resolve...
My version is also 0.37.2, I list my related environment setting here. Python 3.8 bitsandbytes 0.37.2 CUDA 12.1 Transformers 4.28.0.dev0 peft 0.3.0.dev0 torch 2.0.0+cu117
> multi GPUs? IF SO, change device_map="auto" in utils.py line 353 to device_map={"":0} or other index, and add it to PeftModel.from_pretrained(). Thanks, we do not attempt to utilize multiple GPUs...
You need to `cp checkpoints/7b/checkpoint-200/pytorch_model.bin checkpoints/7b/adapter_model.bin` and the lora_model path set as `../checkpoints/7b`