vtddggg

@FzuLJ @842430478 I have this problem too. Have you solved this problem? Hope I can get some advice.

@842430478 @FzuLJ Which proposals method do you use? Is this https://github.com/jponttuset/mcg the mcg which you refer?

@xcpeng @tfwang96 Also get 71.5% zero-shot acc on cifar10. Have you solved this problem? I suspect that the author may use a different data preprocess for cifar10.

Yes, it exceeds 16GB even using `batch-size=1`. Unfortunately, I do not have a GPU with 32GB memory 😂 Maybe I should think other ways to reduce the memory consumption.

"loss is nan" is a problem for original ViT models when amp is turned on. Check [here](https://github.com/facebookresearch/deit/issues/29) for more details. Afterwards, many techniques are proposed to solve this problem, e.g.,...

> @jlindsey15, logically it should work. But may just take too much time to run on ImageNet. > > One note is that currently I am not using `DistributedDataParallel` which...

+1 and when I turn off the masking, the fine-tuned result is terrible. I can only get 79% on ImageNet.

actually I use the [official code](https://github.com/facebookresearch/mae) to reproduce the paper result and below is my log for pretraining ``` {"train_lr": 7.974358974358974e-05, "train_loss": 0.9499255213886499, "epoch": 0} {"train_lr": 0.00023974358974358984, "train_loss": 0.7823955785029401, "epoch":...

Can I ask for your WeChat number for more quick and convenient communication? Wish you can give me more advices for the reproducing. BTW，would you have any plan to open...

> Thanks I can first try on my machines using your code instead of the [official code](https://github.com/facebookresearch/mae) I will post the result when finish training