Phil Wang

@nooralahzadeh yea of course :) you are no longer constrained to a 2 dimensional grid, the sky is the limit here

@nooralahzadeh ohh, if you mean whether the current framework allows you to do that, nope it doesn't, but it's like a 5 line code change to make it do so

@basamelatex no one has shown that this can work with a straight encoder yet afaik, but people have discretized the pixel space and then used a decoder to generate the...

@Alexanzhuo Hi Alex, you won't see any positive results in the small data regime. What you can do, however, is to run self-supervised learning (BYOL) on a bunch of unlabelled...

@xinmiaolin hi, it looks like your training run diverged early the loss should go down to around 0.05 before the images come into view how high is your learning rate?

@xinmiaolin do you mean `3e-4` because `4e-3` is absurdly high!

@xinmiaolin would recommend `1e-4`

@xinmiaolin yea, the loss should be the MSE, which is average across image samples

> The learning rate is 4e-3. Yes, the training loss does drop very precipitously from around 1 to 0.1 in the first epoch, then sharply increased to 0.8 again. Then,...

@xinmiaolin you'll definitely see something DDPMs are so much easier to train than their predecessor, GANs