Dot grid in reconstructed images

[Johnathan-Xie]

Hi, I was replicating your training, but noticed that there appear to be a grid of dots in many reconstructed images even after around 200k steps of training. Did you observe this as well? I assume it has something to do with the patching of the ViT decoder, but I just wanted to make sure this is normal to observe early on in training and that it will eventually disappear.