pt.darts icon indicating copy to clipboard operation
pt.darts copied to clipboard

Is it proper to use valid_loader in both train and validation?

Open kamata1729 opened this issue 5 years ago • 2 comments

I found that valid_loader is used in train(train_loader, valid_loader, ...) and validate(valid_loader, ...) . The gradient of model params (indicated as $\alpha$ in original paper) are calculated with valid_loader in train func, so validation with valid_loader is not fair, I think.

kamata1729 avatar Oct 01 '19 11:10 kamata1729

I think you can view the issue of the official DARTS version, the same question was raised. And the author of the original paper replied that DARTS' s main validation criteria is on the ImageNet instead of CIFAR10

hyqyoung avatar Nov 07 '19 13:11 hyqyoung

in search.py train and valid are splits of train dataset from torchvision. in augment.py train is train from torchvision, valid is valid from torchvision. The later is newer used for architecture search, only for architecture evaluation and thus DARTS is correct.

IlyaTrofimov avatar Nov 07 '19 18:11 IlyaTrofimov