self_supervised icon indicating copy to clipboard operation
self_supervised copied to clipboard

Published 20 hours ago • verified publisher icon KeremTurgutlu

Is it possible to train on multi GPUs?

Open robeson1010 opened this issue 3 years ago • 2 comments

Is it possible to train the model with multi GPUs? It looks can't work with learn.distrib_ctx():

robeson1010 avatar Dec 08 '21 18:12 robeson1010

Could you provide more context about your training script? A link to the script with library versions and the expected error message would be helpful for debugging.

KeremTurgutlu avatar Mar 05 '22 05:03 KeremTurgutlu

@robeson1010 . The following steps helped me to train model with multi GPU support.

  1. Try to import from fastai.distributed import *
  2. Add the following lines at the end of the code
with learn.distrib_ctx(sync_bn=False):
    learn.fit_flat_cos(100, 3e-4)
  1. Copy all the copy and paste them in a python script file (i.e. scriptname.py )
  2. Run the file using following command python -m fastai.launch scriptname.py

Theivaprakasham avatar Mar 29 '22 19:03 Theivaprakasham