TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
## Description Hello, I'm trying to set the precision of specific layers to fp32, but after setting some layers, I don't see any improvement (the final output is still NaN)....
## Description I am now installing TensorRT by following the Readme.md instructions. However, when running make, it fails with error below: ``` error: no instance of function template "cuda::std::__4::plus::operator()" matches...
I am trying to do a QAT training with the simple mobilenetv3-like model. The training goes well, but when I save the model to the Keras and then convert it...
I updated my TensorRT version to 8.5 and got warnings about deprecated functions, opened question on NVIDIA developer forums [tensorrt-8-5-depecated-functions](https://forums.developer.nvidia.com/t/tensorrt-8-5-depecated-functions/278907). I used [void nvinfer1::IRuntime::destroy()](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-801/api/c_api/classnvinfer1_1_1_i_runtime.html#ab07e802f58331e0c7e215e58422bd936) and it `Deprecated interface will be...
TensorRT-8.6.1.6+cuda12.1 GPU RTX3090 24G
## Description Hi, I am working on a model that employs a modified attention mechanism, incorporating pooling on top of K and V to reduce computational load. However, I'm encountering...
## Description I customized TensorRT's Col2Im plugin, recompiled the source code of TensorRT8.5, and generated a new nvinfer_plugin library. This is the LayerNormalization node information in the model,  So...
## Description I convert groundingdino from torch to tensorrt on A100, which can accelarate 50% on inference. However, when I deploy the same model on T4, after I rebuild engine,...
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#fusion-types says: **Depthwise Separable Convolution** `A depthwise convolution with activation followed by a convolution with activation `**may sometimes**` be fused into a single optimized DepSepConvolution layer. The precision of both...
We have exported onnx through the script provided by the enthusiastic experts. The onnx file takes about 5 seconds to infer an image using CPU, and 2 seconds to infer...
