Menshykov

@oleg-trott OMFG! That's exactly what I've been researching for the last month! I've looked for similar papers a month ago, but couldn't find any. I've used these things half a...

So now that it's clear how important groups are, I hope Neon devs will prioritize this higher.

https://github.com/soumith/cudnn.torch/blob/master/SpatialConvolution.lua So the way Torch's cudnn module works is actually the same as I described, launching few kernels consecutively. So it turns out that that wasn't a very good idea....

@oleg-trott To my understanding both gemm and dot is not any good for convolutions, especially on GPU (but I may be wrong.. in that it may be even worse on...

Regarding profiling, with these guys in the paper cutting convs into 64 groups without even making them wider, I guess, no more profiling is required, right? Seems fairly obvious that...

@oleg-trott If by other uses you mean for example grouped linear, I'm totally subscribing. =) Regarding im2col, yeah, that's what I'm talking about. When I was searching for a framework...

On the other hand, for a CPU, maybe, im2col would be relatively appropriate for non-, or little grouped convs at least for large feature map sizes. Needs testing and comparing...

@oleg-trott regarding an Intel company making optimal kernels, one funny reason to do that would be to cut demand for more and higher-cost-margin GPUs by making fewer cheaper ones "good...

Turns out grouped convs are available on TensorFlow as depthwise_conv2d https://www.tensorflow.org/versions/r0.10/api_docs/python/nn.html#depthwise_conv2d . Fully-grouped only, though.

https://arxiv.org/pdf/1611.05431.pdf So grouped convs are now officially the next step in ResNet.