Menshykov

Wow, okay. Guess that would take editing plot_training_log.py.example then.

Actually, easier. Just measure a single forward-backwards pass on the side and then multiply by #epochs from the log. Having info whether the things converged or you just cut it...

It's interesting to find out how ResNets perform for different activations. PReLU was coauthored by Kaiming He - coauthor of ResNets, but for some reason he used ReLUs in ResNets.

Only the one you can find on the main page.

Yes, it's that. Something like ` [input, output] += blockIdx.z * [input, output]_STRIDE (1 is what comes to my mind. But that's naive C and it also makes an assumption...

So I ran a test with original WideResnet 2-40 and a modified version that has 3 residuals instead of 6, but has them in 2x groups combined with a 1x1...

So I ran my experiment and got to ~25% error so far on that test (390 epoch, ~0.002 LR at the moment), which is better than the published 26% for...

Regarding contributions, IMHO, Neon could become a framework of choice for people exploring new concepts that require writing custom CUDA code due to PyCUDA and more overall cleanliness vs Torch/TensorFlow/Caffe....

I may be wrong, because I'm not into SASS, but as far as I understand, all you have to do is just like a 64 bit add to the register...

If it's far from being that easy, I imagine, one could just launch Gnum of kernels consecutively to get Gnum of groups and only sync after they're all completed, though...