minimal-llama
minimal-llama copied to clipboard
zphang
I am new to NLP and currently exploring the LLaMA model. I understand that there are different formats for this model - the original format and the Hugging Face format....
I see that : def model_forward(model, inputs): h = inputs h = h.to(model.base_model.model.model.embed_tokens.weight.device) h = model.base_model.model.model.embed_tokens(h) for layer in model.base_model.model.model.layers: h = h.to(layer.input_layernorm.weight.device) h = layer(h)[0] h = h.to(model.base_model.model.model.norm.weight.device) h...
Hi I can load the model fine via model = transformers.LLaMAForCausalLM.from_pretrained("/content/drive/MyDrive/llama-13b-hf/") but Im not finding the LLaMATokenizer, so receiving the error AttributeError: module transformers.models.llama has no attribute LLaMATokenizer
When running the finetuning with peft with the command: `python finetune_peft.py --model_path ../../LLaMAHF/llama-7b/ --dataset_path ../../tokenizedinstruct/ --peft_mode lora --lora_rank 8 --per_device_train_batch_size 2 --gradient_accumulation_steps 1 --max_steps 30000 --learning_rate 2e-4 --fp16 --logging_steps 100...
Your model does not seem to be able to calculate the gradients of the layers correctly. When I run finetune_pp.py and print the loss during training, after the first optimizer...
Hello, could you please elaborate on what "Seems buggy, don't use this yet." means for the 8-bit + pipeline parallel example? What bug is there specifically? Does it affect training...
I have finished training the model and would like to use it for inference. Could you please tell me what "pefg_path" means? Thank you very much.
Is it possible to run minimal-llama fine tuning on Windows without bitsandbytes?
`zphang/transformers` is required before running `tokenize_dataset.py`.
