Feature Request: integrated depthwise then pointwise convolutions?

[bsugerman]

This is an awesome repo, Thanks for making it! One of the annoyances of pytorch is that if you try to use a 3x3 depthwise conv and then 1x1 pointwise conv to use fewer params than a straight 3x3 convolution, the memory requirement explodes and the model actually slows down. I'm curious whether you'd be interested in fusing these two convolutions as a single layer in cuda as part of your repo? There is a lot of interest on the pytorch boards (see https://discuss.pytorch.org/t/using-optimised-depthwise-convolutions/11819/15 for example).

[rosinality]

It will be nice to have, but as in many cases batch norm is used between depthwise and pointwise conv, maybe the application could be limited.

[bsugerman]

Leaving out the batch norm between those two layers is a price I'm very willing to pay. I think you'd be quite the hero if you added that layer.

[rosinality]

Yest, it can be worth to try. I will look at it.