stanford_alpaca

stanford_alpaca copied to clipboard

longshuicui

configuration：Ubuntu2204，200G RAM，24 core（Xeon E5-2630 v3 @ 2.4GHz）, no GPU. I picked out 10000 records from alpaca_data.json for training. 121 hours have been used. got an unrecognized response. when i used...

gugongerguo

tetratorus

SeekPoint

SFT Mistral；

2

torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

feiying12343

ValueError: Trying to set a tensor of shape torch.Size([32769536]) in "weight" (which has shape torch.Size([32001, 4096])), this looks incorrect.

3

When I finetune llama7b: ``` # alpaca torchrun --nproc_per_node=8 --master_port=29000 train.py \ --model_name_or_path .cache/hub/models--meta-llama--Llama-2-7b-hf/snapshots/01c7f73d771dfac7d292323805ebc428287df4f9 \ --data_path ./alpaca_data.json \ --bf16 True \ --output_dir output/Llama-2-7b-hf \ --num_train_epochs 3 \ --per_device_train_batch_size 2 \...

daidaiershidi

In the README on GitHub, it states: "Addressing OOM - Naively, fine-tuning a 7B model requires about 7 x 4 x 4 = 112 GB of VRAM." Could you please...

mangyeong

Million thanks to your great work! Could you help me with my problem? After I finetune my llama model, when I prompt it with text like "Below is an instruction...

starrlee356