qlora
qlora copied to clipboard
artidoro
QLoRA: Efficient Finetuning of Quantized LLMs
Aims to fix #38 and #41 Currently, we get extremely small adapter files on checkpoint. This seems to be due to some issue in the PEFT library. One of the...
max_memory=max_memory, NameError: name 'max_memory' is not defined For testing, running on a smaller machine, 128GB, 48GB GPU ram Maybe the example could use a suggestion eg... max_memory={0: "44GiB", "cpu": "110GiB"})
`python qlora.py --model_name_or_path decapoda-research/llama-13b-hf` (I have updated the tokenizer_config.json and config.json as per the various discussions [here](https://huggingface.co/decapoda-research/llama-13b-hf/discussions/) `tokenizer_class`: `LlamaTokenizer` and `architectures`: `LlamaForCausalLM`) ================================================================================== adding LoRA modules... trainable params: 125173760.0 ||...
The saved adapter_model.bin is only 441kb. https://github.com/artidoro/qlora/issues/38
Check for LlamaTokenizerFast rather than infer type from path name. Fix cases where non-standard llama model path names gets bypassed in tokenizer check. The tokenizer is init with use_fast=True and...
I am using the `finetune.py` script with default params it works perfectly but when I get the adapter model in the checkpoint directory, the size of adapter model is just...
---------pip package version transformers-4.30.0.dev0 accelerate 0.20.0.dev0 bitsandbytes 0.39.0 peft-0.3.0.dev0 ----------python cmd python qlora.py --model_name_or_path /home/bmb/models/facebook/opt-125m ---------error Traceback (most recent call last): File "/home/bmb/projects/qlora/qlora.py", line 766, in train() File "/home/bmb/projects/qlora/qlora.py", line...
Trained vicuna-13b-1.1 LORA in 4bit Now trying to merge it for running generations but it fails with the following error ``` python3.11/site-packages/peft/tuners/lora.py", line 352, in merge_and_unload raise ValueError("Cannot merge LORA...
I'd like to fine-tune using unlabelled data, i.e. a causal language modeling. For instance to adapt a model to a new domain or language. Which parts of the training code...
