question about training time in classic_cv and use_amp

[wyyfffff]

First thanks to your code base!

It needs to be acknowledged that your USB_cv's ViT backbone is excellent and can get better result in shorter time than classic_cv, but I think using pretrained weight is a bit like transfer learning rather than SSL. So after using ViT backbone I go back to train with WRN backbone, and I found it takes to much time to train, it takes almost 10 days to train fixmatch with RTX6000 (and it takes 8 days to train in provided log), so is there any methods to reduce the training time without reducing accuracy?

Another question is that, the training log before and after I set use_amp=True are same, I really don't know why.

Thanks!

[qianlanwyd]

I think SSL in CV should not exclude pre-trained models as they are widely used in NLP and Audio tasks. SSL and transfer learning are not mutually exclusive.

For the training cost, the classic training will take 1,000,000 steps for convergence. We cannot reduce the training time easily.

We actually do not use amp in all experiments. For the use of amp, you can check Semi-supervised-learning/semilearn/core/hooks/param_update.py to see whether it is correctly used. If there is any bug, please tell me.