Not getting desired performance on rotmnist

[dkgupta90]

@jatentaki thanks for the code. I was wondering if this code achieves similar performance with hnet on rotmnist. Compared to the work of Worrall et al where they get 98.5%, I get only around 94% with this code. Could you please let me know what could be going wrong?

[jatentaki]

Sorry for late reply. In short, I can restate what I said via e-mail: this code never attempted to reproduce the work of Worrall et al. exactly, it is merely a byproduct of my MSc thesis which had some additional requirements/considerations. I have never obtained 98.5% performance with this code myself. One obvious difference is in the activation function which is different from what Worrall implements. Unfortunately, I can't devote the time to trying to reproduce their numbers exactly.

Regarding your comparison of results with a standard CNN and harmonic networks: I don't know where you are taking your numbers from but please note that the MNIST example is meant to be trained with MNIST (and not ROTMNIST) data and then tested on ROTMNIST. This means that the reasonable comparison is between python main.py --model hnet --train mnist and python main.py --model baseline --train mnist. This way we can see that even though the harmonic network never sees the rotated digits during training, it still generalizes to them at test time, unlike a regular CNN. If this is not what you find, please let me know.