stanford_alpaca
stanford_alpaca copied to clipboard
tatsu-lab
Code and documentation to train Stanford's Alpaca models, and generate the data.
To execute locally ```bash pyflyte run train.py train --model_args='{}' --data_args='{}' --training_args='{"output_dir":"/tmp"}' ```
Please correct me if I got anything wrong. I am trying to learn more about LLM research. Alpaca Contribution: Your research team instruction tuned llama via the self-instruct method and...
when I use two A100 nodes, each node is (80GX8). I found two nodes train is slower than one node. I use torchrun xxx. can any one meet this?
Hi everyone, I tried to reproduce the finetuning of the alpaca, but I met follow error. Could you please help me? ```python Running command git clone --quiet https://github.com/huggingface/transformers /tmp/4267942.1.nvidiagpu.q/pip-req-build-317x2j5l ERROR:...
Getting this error while using single A100 8G0GB while loading llama-7b I tried reducing the batch size also changes the **--gradient_accumulation_steps** but not able to work it out. I was...
We propose a new learning paradigm named RRHF (Rank Responses to Align Human Feedback) which does not need reinforcement learning and can perform on par with PPO to align human...
ERRO: ``` Using .cache/torch_extensions/py310_cu117 as PyTorch extensions root... Emitting ninja build file .cache/torch_extensions/py310_cu117/utils/build.ninja... Building extension module utils... Allowing ninja to set a default number of workers... (overridable by setting the...
I use the 8 v100 train the model, the saved model is error, the size of model is : the command is as follows torchrun --nproc_per_node=1 --master_port=12345 train.py \ --model_name_or_path...
I cannot start running the train.py script (on 2 x 4090 gpu) Got this error: File ".../alp/lib/python3.10/site-packages/transformers/hf_argparser.py", line 341, in parse_args_into_dataclasses raise ValueError(f"Some specified arguments are not used by the...
