pfeatherstone

Yeah that paper is nice. I like these new simple building blocks. I hope they do actually work well beyond simple resnet modules.

Can this be implemented in the toeplitz matrix ?

Just noticed the [toeplitz matrix](https://github.com/davisking/dlib/blob/5cb036564c986fcc45eb16dd0174b1602ceb4f7f/dlib/cuda/cpu_dlib.cpp#L2536) isn't cached in the `tensor_conv` class. So there are a lot of allocations and deallocations. Right?

@davisking OK, I trust you. I had a quick look at the [ncnn](https://github.com/Tencent/ncnn) repo, and they have like a 100 specialisations of the convolution layer for different parameters like kernel...

You can also train in pytorch, convert to onnx and infer using onnxruntime which is really fast, easy to use and has loads of execution providers.

I'm surprised ```std::tuple_size``` works with ```std::array``` and ```std::pair```

Yep, apparently it's in the standard. gosh

Rather than: ``` template< typename T > struct is_reference_wrapper : std::false_type {}; template< typename U > struct is_reference_wrapper : std::true_type {}; template auto INVOKE( T Base::*pmf, Derived && ref, Args...

I've put both implementations in compiler explorer, turned optimizations to -O2, and you get exactly the same assembly. So i think you get the same level of inlining

Anybody have any opinions on this? I am interested in people's thoughts.