Reproducing results

[rheasukthanker]

Hi,

Thank you for the great work and for open sourcing your code!

I have tried reproducing the results for NB201 and the DARTS search space.

On the NB201 search space using cd 201-space && python train_search.py, I can reproduce the results from your paper as follows:

• cifar10 : 94.360000
• Imagenet16-120 : 46.340000

For CIFAR100 on NASBench201 search space however I could only obtain 70.88 contrary to the reported 73.51. On the DARTS search space when searching on CIFAR-10 using cd DARTS-space && python train_search.py I am not able to obtain the same genotype as the one mentioned in the repo. With the new genotype I get an error of 2.89±0.091.

Secondly when performing evaluation on the DARTS search space using cd DARTS-space && python train.py --cutout --auxiliary and the DrNAS_cifar10 genotype from the repo, I could obtain an error of 2.67±0.090 which is higher than the 2.46 ± 0.03 reported in the paper.

Any help in replicating the results on the DARTS search space would be greatly appreciated! Thanks!