deepvats icon indicating copy to clipboard operation
deepvats copied to clipboard

Speed up model inference with Nvidia TensorRT

Open vrodriguezf opened this issue 3 years ago • 0 comments

I just saw this a couple of days ago: https://developer.nvidia.com/blog/nvidia-announces-tensorrt-8-2-and-integrations-with-pytorch-and-tensorflow/

It seems to be an "easy" way of speeding up inference time in Pytorch/TF models with one line of code. This would be critical for the use of the tool, since right now the bottleneck lies in the computation of the embeddings.

vrodriguezf avatar Dec 13 '21 11:12 vrodriguezf