stanford_alpaca
stanford_alpaca copied to clipboard
tatsu-lab
Code and documentation to train Stanford's Alpaca models, and generate the data.
This will be insane....
Great work, this is a very exciting direction! In addition to the raw text data in [alpaca_data.json](https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpaca_data.json), are you able to release the token probabilities generated by GPT-3 for each...
Hi, can anyone help me about it. I tried to use a small set of official dataset for training. However, the train loss is quite high. And the responses which...
[Mar20_05-17-08_0c56f6779a08.csv](https://github.com/tatsu-lab/stanford_alpaca/files/11024692/Mar20_05-17-08_0c56f6779a08.csv) Training command ``` torchrun --nproc_per_node=4 --master_port=34322 train.py \ --model_name_or_path {your-hf-lamma-path} \ --data_path ./alpaca_data.json \ --bf16 True \ --output_dir {your-output-dir} \ --num_train_epochs 3 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \...
I am blocked at this step.Seems it asked me to choose wandb and it's stucked after I type `3`. No progress.. Is this expected? ``` root@5d83a2b86756:~/stanford_alpaca# torchrun --nproc_per_node=4 --master_port=3192 train.py...
Would PRs to `main` fixing errors in the `output` field of data set items be welcome? If so, the readme should clearly state that the data set is work in...
Hi, thanks for your great work! which transformers did you use for training? I tried to reproduce the result with transformers 4.28.0, but I got the following error: ` __init__()...
What is the `fsdp_transformer_layer_cls_to_wrap` for bloom? When I tried to fine tune with bloomz-7b1, the training stuck on 0%. As you said in the readme, it's most likely because I...
