narrowsnap
narrowsnap
I reproduce ViT on CUB using the reference https://github.com/rwightman/pytorch-image-models, and got an accuracy of 91.06%(with vit_base_patch16_384(pretrained=True, num_classes=200, drop_rate=0.1), optim.SGD(model.parameters(), lr=0.0001, momentum=0.9, weight_decay=1e-5), lr_scheduler.ReduceLROnPlateau(optimizer, 'max'), run 100 epoch)
What is your model config? I meet the same error because I don't set the data_preprocessor in model.
In my experiment, resnet50 got 88.23% accuracy, and resnet101 is worse than that.
@kabachuha @patil-suraj Is there any update?
In the DiT training code, the purpose of setting sigma is to learn the mean and variance of the noise, and then calculate the KL loss with the gold standard....
> The model released is also 8 channels (including mean and variance). Why set the model to predict mean and variance, but do not calculate the loss of variance. I...
> I felt the same doubt. why Just use out_channel=8 ,uses the mean (the first 4 channels) ,dorpout bias? Maybe,Update latent_z by grad without bias > > ``` > dt...
In my experiments, I change it to `master_param = optimizer.working_to_master_param[param_id]`.
> After changing it to `master_param = optimizer.working_to_master_param[param_id]`, I still have the `AttributeError: : 'LowLevelZeroOptimizer' object has no attribute 'working_to_master_param'`. Can anyone help? What's your version of colossalai?
> caption_emb_masks Are you update the code of RFLOW?(opensora/schedulers/rf/\_\_init\_\_.py)