Contrastive-Clustering icon indicating copy to clipboard operation
Contrastive-Clustering copied to clipboard

weight_decay

Open YIBO-0805 opened this issue 3 years ago • 5 comments

Why the parameter 'weight_decay' set in code is 0.

YIBO-0805 avatar Oct 26 '21 02:10 YIBO-0805

We didn't tune that hyper-parameter during the experiments. Perhaps a proper weight decay could further improve the performance.

Yunfan-Li avatar Oct 26 '21 02:10 Yunfan-Li

But I get only 72.3% accuracy on cifar10 dataset. I run it in pytorch1.9.0+cuda11.4.

YIBO-0805 avatar Oct 26 '21 03:10 YIBO-0805

Maybe it is due to the different environment and random seed. The config we provided is exactly the one we used to produce the result reported in the paper. You may check the code again to make sure no additional modification is made and rerun the code with a different random seed.

Yunfan-Li avatar Oct 26 '21 03:10 Yunfan-Li

We didn't tune that hyper-parameter during the experiments. Perhaps a proper weight decay could further improve the performance.

What might be an appropriate weight decay for CIFAR10?

mark375chen avatar Feb 01 '22 21:02 mark375chen

We didn't tune that hyper-parameter in our experiments. You may try the commonly used 1e-3 or 1e-4.

Yunfan-Li avatar Feb 02 '22 14:02 Yunfan-Li