gpt-2 icon indicating copy to clipboard operation
gpt-2 copied to clipboard

how to train on multi gpu

Open brianjcj opened this issue 5 years ago • 1 comments

how to train on multi gpu?now it only use the first gpu by default

brianjcj avatar Aug 14 '19 15:08 brianjcj

Add

import os
gpu_id = "3"  # the GPU id you want to use - also multiple GPUs possible (separated by comma)
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = gpu_id

to train.py

(source https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/)

Best regards

EndruK avatar Sep 10 '19 08:09 EndruK