Reproducing nb201 results

[gurizab]

Hi,

Thank you for the great work!

I was trying to reproduce the nb201 results, however, while I was able to reproduce the results for Cifar10 and Imagenet16-120, I couldn't do the same for Cifar100. I am running the train_search.py file, and just changing the dataset argument to run.

The results I am getting for 100 epochs using the default hyperparameters provided in the script are:

• NB201 test accuracy cifar10: 94.36%
• NB201 test accuracy cifar100: 70.47%
• NB201 test accuracy imagenet16-120: 46.34%

Any help would be appreciated!

[xiangning-chen]

Hi,

Thanks for your interest, which pytorch version are you using?

[gurizab]

Hi, I am using torch 1.3.0+cu92 and torchvision 0.4.1+cu92

[xiangning-chen]

I suppose that the version should not cause this issue, as we can get the CIFAR-100 result with 0 variance. Could you also try train_search_progressive.py?

[gurizab]

Hi again, Even after trying train_search_progressive.py I still could not reproduce the cifar100 results. The result that I got is: 70.01%