MultiGPU support ETA?

[StrangeTcy]

I'd like to finetune my own model with your code, but the DalleModel as it is now doesn't seem to support the pytorch DistributedDataParallel wrapper. Do you plan to add such an option?

[shonenkov]

@StrangeTcy good evening!

finetuning example on free colab you can find here: Finetuning example

it doesn't support DDP, but feel free add these features

we would like to share full training code after publish paper :)