multi-GPU support

[tmabraham]

I need to check if fastai's multi-GPU support work with my package, and if not, what needs to be modified to get it to work. Additionally, I may need to add a simpler interface for DDP or at least clear examples/documentation. This will enable for quicker model training on multi-GPU servers, like those at USF.

[tmabraham]

I will probably wait until fastai+Accelerate support is in a good state (see fastai Discord for current status) and will see if I need to add anything for support then...

[FireWallDragonDarkFluid]

Hi, really nice project, learn a lot from your work!!!

Just wondering, now that fastai2.7 has made some improvement on distributed training, are you planning on adding that to the project?

Or perhaps demo a snippet of code showing how to integrate GAN written in fastai with that functionality? Since there is no example showing how to achieve that.

Thanks a lot.