One RTX 3090, 88 hours, is this normal?

[kli-casia]

python finetune.py --base_model='decapoda-research/llama-7b-hf' --num_epochs=10 --cutoff_len=512 --group_by_length --output_dir='./lora-alpaca' --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' --lora_r=16 --micro_batch_size=8

[andreae293]

i get 118 hours on A4000 with the following command, so i guess its normal

python finetune.py --base_model='decapoda-research/llama-7b-hf' --num_epochs=10 --cutoff_len=512 --group_by_length --output_dir='./lora-alpaca' --lora_r=16 --micro_batch_size=8

[cxfcxf]

i think epochs=10 means it will train with the same datasets ten times.

[CZWin32768]

I tried the code with two 3090 cards and the estimated end time is ~+35hours.