No accuracy boost with default settings

[agnonchik]

I don't observe any accuracy boost for dy_resnet18 compared to raw_resnet18 model on CIFAR10 and CIFAR100 datasets. I used default hyper-parameters for the training and observed. CIFAR10: dy 86.01%, raw 87.75% CIFAR100: dy 54.59%, raw 56.52%. All the experiments were bellow my expectations (>90% for CIFAR10, >70% for CIFAR100). How to take advantage of dynamic convs?

[RongKaiWeskerMA]

The implementation looks fine according to the paper. But not sure if there are tricks to make the algorithm work. Or maybe it is not even working? Idk.