Phil Wang

> @lucidrains no, unfortunately I get this error: `RuntimeError: Given groups=1, weight of size [768, 3, 16, 16], expected input[2, 1, 32, 1024] to have 3 channels, but got 1...

@marianna13 oh great! can you confirm that you are using 1 channel then? i should revert that commit

@marianna13 i'll add the `MulanCoCa` version tomorrow too, so we can possibly leap frog the state of the art going on within google

> @lucidrains yes, I changed back to 1 channel and it worked, but also I tried to run it over a batch of images but it didn't work :( oh,...

@marianna13 if you can show me a reproducible error like the sample script above, i can fix it

@marianna13 hey Marianna, thanks for sharing the script it looks good except for the image dimensions, whose height and width needs to be divisible by the patch size. however, that...

@marianna13 hey Marianna, were you able to do a small test run? if we can even get a training run to overfit on a small training set, maybe we can...

@havu73 hello! while the network was trained at 191k, you can still run it at 131k (or even less, provided it is divisible by the total downsample factors of the...

@conceptofmind Hi Enrico again and thank you for running this experiment Was the above run done in f32 or f16?

@conceptofmind ohh yes the fp16 is the likely cause, as i was in the middle of fixing an underflow issue with the way cosine sim attention was approached in the...