magvit2-pytorch icon indicating copy to clipboard operation
magvit2-pytorch copied to clipboard

recon images is black

Open syyxsxx opened this issue 1 year ago • 9 comments

i train magvit2 on imagenet the recon images is black, but the loss looks normal sampled 18 (1) image

syyxsxx avatar Jan 16 '24 09:01 syyxsxx

I met the same problem. Have you solved this?

nangying1112 avatar Jan 19 '24 03:01 nangying1112

I met the same problem. Have you solved this? I also encountered the same problem where the loss remains constant, and I discovered that the gradient is NaN.

dyflional10 avatar Jan 19 '24 08:01 dyflional10

I turned off mixed precision training,solve this problem

syyxsxx avatar Jan 22 '24 15:01 syyxsxx

@syyxsxx thank you for finding and sharing the source of the issue. i'll figure out what's the matter with mixed precision..

lucidrains avatar Jan 23 '24 16:01 lucidrains

@lucidrains Thanks for your great work. I turned off mixed precision training, the loss looks normal, but the image reconstruction is abnormal. sampled 273 截屏2024-01-24 11 00 03

dyflional10 avatar Jan 24 '24 03:01 dyflional10

I turned off mixed precision training,solve this problem

can you share your config and training log, thanks

misery0424 avatar Jan 24 '24 12:01 misery0424

I also met the same problem the construction image is not so normal. Can you share the details of solving this problem? How to turn off the mixed precision traning

I turned off mixed precision training,solve this problem

VoyageWang avatar Feb 02 '24 09:02 VoyageWang

我在 imagenet 上训练 magvit2, 侦察图像是黑色的,但损失看起来很正常 采样 18 (1) 采样 18 (1) 图像

hello,Have you solved this problem?

J-Wu97 avatar Feb 26 '24 07:02 J-Wu97

Hi, I have a question. Why use images to train magvit-2? Should it designed for videos?

kxgong avatar May 16 '24 08:05 kxgong