Angainor Development
Angainor Development
Gradio magic, yep, comes with a "free" API, just need proper setting of functions to make visible.
I'm training 7b and 13b on 3090. This repo is supposed to load the models with 8bits, so 13B model = 13GB vram on the GPU. Your error seems to...
Are you sure you used the right (model, peft) couple? Check your peft config json , `adapter_config.json` to make sure you used the correct base model for inference.
This model is not a standard one, it comes with its own tokenizer https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b/tree/main and has a dedicated repo with related doc https://github.com/ymcui/Chinese-LLaMA-Alpaca The error is likely related to the...
1 single epoch is too low. First gen rank 8 loras were using 3 epochs. Current params with 4 lora modules and rank 16 use closer to 10 epochs. Also,...
See just below https://github.com/tloen/alpaca-lora#official-weights --group_by_length however I do not recommend.
The tokenizer warning you can ignore. Since you have mixed gpus, I'd begin with trying with one kind of GPU only at a time, see what it gives.
How many gpus? What command line did you use? (full params please)
Apart from the --group_by_length param, it's very similair to how I run it, and your batch params seem consistent. Loss at 0 definitely is weird. I'd try clearing the .hf...
Yeah, 10 items is not a training dataset. Way too few tokens to move the lora weights, unless you train for long enough and then break the model.