How to reproduce the results on CIFAR10

[zhangshuaitao]

How to reproduce the results on cifar10, I downloaded the net.config and net.init files of cifar10 in the table, and ran the valid script, but only got top1=10.

[zhangshuaitao]

@mikelzc1990

[mikelzc1990]

There's an inconsistency between search dataloader and validation dataloader, please use the following commend as a quick hack for now. You will also need to comment this line.

I'm traveling right now, will update the code with a proper fix next week.

python train_cifar.py --data /mnt/datastore/CIFAR/ --dataset cifar10 --evaluate --model-config ../net.config --initial-checkpoint ../net.init

[zhangshuaitao]

@mikelzc1990, I have fixed the bug that the pre-trained model is not loaded in the code, and got Acc@1 97.7 by using net.config and net.init in net-flops@468. I guess that the current net.init is the weight obtained during the network search. I need to use train_cifar.py to fine-tune on cifar10 to get the Acc@1 98.4 reported in the paper. Is that true?

[AwesomeLemon]

The code seems to have not been updated to fix this. The main problem is not loading the pretrained weights (thanks @zhangshuaitao for mentioning that), which can be fixed by adding the following 2 lines to validation.py here (https://github.com/mikelzc1990/nsganetv2/blob/0e68ac3a4f/validation.py#L76)

    init = torch.load(args.pretrained, map_location='cpu')['state_dict']
    model.load_state_dict(init)