TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
## Description I use onnx-graphsurgeon to merge gelu ops, and rename Node in graph is "CustomGeluPlugin". I have also replaced plugin name from "CustomGeluPluginDynamic" to "CustomGeluPlugin". But when I use...
## Description When I use the GELU Plugin in my project, it increases the inference time. **Before** GELU Plugin, inference time is **44ms** (fp32). **After** GELU Plugin, inference time is...
We converted the ONNX model with IsInf OPs using TPAT and it succeeded. We noticed that the IsInf OP is implemented by tpat_ininf and Cast OP. When we convert the...
hello, when i coverted my onnx model to TensorRT by the command, `./trtexec --onnx=model.onnx --saveEngine=model.engine` i got big diff between pytorch result and trt result. i located the problem which...
## Description I am trying to use the ubuntu 20.04 aarch64 dockerfile (cuda 11.4 one) it gives me the following error: E: Version '8.4.2-1+cuda11.4' for 'libnvinfer8' was not found E:...
## Description when transform googlenet changed by pytorch_quantization from onnx to engine, it reported the error below. I want to ask if there are some experience or some tools can...
## Description The example [here](https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Core/Logger.html) is below ```python from tensorrt import tensorrt as trt class MyLogger(trt.ILogger): def __init__(self): trt.ILogger.__init__(self) def log(self, severity, msg): return MyLogger() ``` Unfortunately, this raises ```...
Hello, when I use tensorrt7.2.3.4, the errors were occured [08/23/2022-11:24:58] [TRT] [E] C:\source\rtSafe\safeRuntime.cpp (32) - Cuda Error in nvinfer1::internal::DefaultAllocator::free: 1 (invalid argument) I don't know why, thanks for your answer
## Description TRT PluginCan it support multiple outputs ,I have implemented a multi-output plugin myself, but it will report an error in the next op node。 -------------------------SgnetAnchorsVpDynamic::getOutputDimensions-----------0 outputIndex:0 ret.d[0]:1539 ret.d[1]:77...
## Description When using resnet50 of the timm model library, when PTQint8 is quantized, the accuracy becomes worse. In the same environment, the accuracy before and after quantization on torchvision...
