Add TensorRT Support.

Open TheBlackPlague opened this issue 2 years ago • 2 comments

It would be fantastic if TensorRT support is added to TorchSharp directly (at least on the CUDA variants). TensorRT performance (due to kernel optimizations) beats regular PyTorch inference performance by magnitudes. It would be great to see this implemented.

https://github.com/NVIDIA-AI-IOT/torch2trt

Mar 16 '23 23:03 TheBlackPlague

@TheBlackPlague

I could see U are super pro with pytorch.

If you were here 6 months ago, @NiklasGustafsson is pretty much ONE person doing close to 100%.

Now we have more community help. We hope more will find ways to un-load Niklas's long list.

Try to use the "Discussion" to keep on sharing where TorchSharp need to go and be

Mar 17 '23 08:03 GeorgeS2019