FindTheTruth

[Question about multi-gpu training]

2

when I try to train NLG model on multi-gpu,I use this: ``` python -m torch.distributed.launch --nproc_per_node=2 --use_env src/gpt2_ft.py \ --train_data ./data/e2e/train.jsonl \ --valid_data ./data/e2e/valid.jsonl \ --train_batch_size 8 \ --grad_acc 1...