yxy123

When use one Gpu do model training, met one issue.

3

bash training/finetune_RedPajama-INCITE-Chat-3B-v1.sh My configurations changes as below: --lr 1e-5 --seq-length 2048 --batch-size 8 --micro-batch-size 1 --gradient-accumulate-step 1 \ --num-layers 2 --embedding-dim 2560 \ --world-size 1 --pipeline-group-size 1 --data-group-size 1 \...