gpt-2
gpt-2 copied to clipboard
how to train on multi gpu
how to train on multi gpu?now it only use the first gpu by default
Add
import os
gpu_id = "3" # the GPU id you want to use - also multiple GPUs possible (separated by comma)
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = gpu_id
to train.py
(source https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/)
Best regards