Using test data in train.py for validation?

[dkumazawa]

Thanks for sharing the beautiful implementation!

If I am not mistaken, it looks like you are using CIFAR10's test data for validation purposes in train.py:

  train_data = dset.CIFAR10(root=args.data, train=True, download=True, transform=train_transform)
  valid_data = dset.CIFAR10(root=args.data, train=False, download=True, transform=valid_transform)

and you are using the test data again in test.py:

  test_data = dset.CIFAR10(root=args.data, train=False, download=True, transform=test_transform)

My question is, how did you choose your model in the training phase (i.e. architecture evaluation phase)? Did you simply take the final model at the very last epoch of training, without any early stopping based on validation (more like test) performances?

[Catosine]

Hello @dkumazaw !

Thanks for your question. I think the test.py is used as a demo code, which only suggests that their method works.

The performance of an architecture should be evaluated on the ImageNet, because the model built on CIFAR10 would have a "natural tendency" of overfitting.

GL,