Corey Lowman

Closing this one as this will be staged into multiple PRs

One benefit of rust is that if we move to channel last, we get errors at compile time if someone uses channels first 😄

I think this can be non-breaking at the `nn` level by doing impl for both channel first and channel last. channel first can do permute before and after & then...

Did some local testing on CPU over a variety of input shapes, and channels last wasn't faster. Channels first is more standard anyway, so will keep that for now

@AntonioLucibello totally agree about something that just works. Agree with jafioti that supporting cuda is a necessity, but I was thinking the other day that it'd be nice to support...

Also I'm working on a prototype using cust (the base crate of rust-cuda), will hopefully share in the next couple weeks. There's going to be a TON of work involved...

For future reference: - tensorflow's .cu kernels are in this directory: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/kernels - pytorch's kernels are in this directory: https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native/cuda

I spent a bit of time trying to prototype something using cust (from Rust-CUDA project), however ran into a number of difficulties. Notably they don't have nvrtc, which is necessary...

Yeah there's a separate issue for onnx support #57, and I think there's actually a rust onnx wrapper (at least for inference). For now I'm going to continue along the...

Yeah this would be a great win IMO. The diff would have so many deleted lines 😁