Train with taming's VQGAN

[WocMaker]

Hi, I was training on my own data set with taming's VQ-GAN. There's some error with the dimension with the vae model Can anyone help me with it? How can I change this part to let it work?

[lucidrains]

@rom1504 could this be related to your recent PR?

[WocMaker]

@rom1504 could this be related to your recent PR? Hi lucidrain, I had checked rom's dalle server respo before. I still have the PR. Do I need to modifiy more, besides adding the args --taming to use VAGAN as vae?

[WocMaker]

Since when I use open AI's vae with my own dataset, the loss is not decreasing.

[rom1504]

@lucidrains maybe. I will check today (and fix if there's indeed a regression)

@WocMaker no taming option is indeed all that is needed.

The results with openAI vae are indeed as expected (and as I measured in my own experiments). It's about 4 to 8 times slower to converge when using openAI vae (mostly due to the 1024 seq length).

[WocMaker]

@rom1504 Thanks, Romain. Maybe there's some problem with my dataset. It's werid that my dataset can run with openAI vae, but it result in dim error for VQ-GAN. May I just ask for double check that, the dataset format needed is the same with these two vae setting up? orz

[rom1504]

hi @WocMaker , I don't think there is a problem with your dataset. There was a recently introduced bug in Dalle-pytorch code, that I fixed in https://github.com/lucidrains/DALLE-pytorch/pull/329 You can directly use this change or wait for it to be merged

[lucidrains]

@rom1504 thank you! merged 🙏

[WocMaker]

@rom1504 @lucidrains Thank you so much!!!