ResNet-18 parameters are much much higher

[abdulsam]

I used pytorch-model-summary library to look at the summary of ResNet-18 model.

from pytorch_model_summary import summary

def ResNet18():
    print(summary(ResNet(BasicBlock, [2, 2, 2, 2]), torch.zeros((1, 3, 32, 32)), show_input=True))
    return ResNet(BasicBlock, [2, 2, 2, 2])

I observed that the number of parameters are much higher than the number of parameters mentioned in the paper Deep Residual Learning for Image Recognition for CIFAR-10 ResNet-18 model. Have a look at the model summary:

Now look at the table mentioned in the paper:

Why the parameters are so high in this implemented model?

[vamshichowdary]

This is because the Resnet implemented in this repo is not exactly the same as original author's implementation. Original author's implementation is more suited for imagenet dataset. I think the closer implementation to the one in paper is in pytorch's repo: https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py

Main differences I observe:

1. First conv layer is of 7x7 kernel size with stride=2 and padding=3 in the original resnet. In the repo its 3x3 with stride=1 and padding=1
2. There is no max pooling layer in this implementation (although this directly doesn't influence the number of parameters, I think it affects them in deeper layers)

[bamert]

Both this and the repo in https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py do not implement Resnet-20 for Cifar10 in the same way as described in Deep Residual Learning for Image Recognition. In addition to 1, 2 mentioned by vamshichowdary, the paper mentions

• "The numbers of filters are {16, 32, 64} respectively". (Here they are 64, 128, 256, 512)

[vgthengane]

Have a look at this https://pytorch-tutorial.readthedocs.io/en/latest/tutorial/chapter03_intermediate/3_2_2_cnn_resnet_cifar10/. It uses the same configuration as mentioned in the Deep Residual Learning for Image Recognition.

[DhDeepLIT]

What about best accuracies when training from scratch ?