Does not learn

[alphamupsiomega]

The mnist code fails to learn anything for at least the first 15 epochs and counting. It is still running. The accuracy is static at about 10%. If the net does start learning, about what epoch does it start? Why is it not learning at least in the beginning?

If this net ever does start learning, it seems at the rate it is learning is significantly slower than a traditional static fixed-size conv net by many orders of magnitude.

[kutoga]

I am currently traveling around and I do not have access to the resulted files with the training accuracies for each epoch. Unfortunately, the training for the MNIST really took some time. Therefore, I have run it on a Titan X.

The following file contains the setup and the result: https://github.com/kutoga/going_deeper/blob/master/doc/going_deeper.pdf

As can be seen on page 12, the accuarcy should be very high after at most 100 epochs. I have run this test multiple times, therefore this result should be stable.

Currently, the performance is really not that great and I guess it is hard to train non-trivial networks on the CPU. The current performance is therefore really much worse than with a static sized network. The current state is really an alpha-version. I already started to implement everything in pytorch, because there it is much easier (and more effective) to handle dynamic growing networks. Probably, soon I will release the first version of the code.

I'm sorry that the current code did not work as expected for you.