TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
When I use **def"_export_onnx_"** to save a onnx model, such as Resnet50 after quantization, I followed the example in the official documentation. But, it is a pity, the following problems...
The comment at https://github.com/NVIDIA/TensorRT/blob/main/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp#L155 says that the input shape is `[B, S, 3*N*H]` or `[B, S, 3*E]`; but the README.md file says the input shape is `[S, B, 3*E, 1,...
I build TensorRT-OSS v8.0.1.6 + TensorRT-8.0.1.6.Linux.x86_64-gnu.cuda-11.3.cudnn8.2,for purpose of writing custom ops, it need python, In my deployment, no python is needed, so I want to remove python related code from...
## Description When I run trtexec, can verbose be saved directly to a log.txt? What parameters should I set? ## Environment **TensorRT Version**: 8.2.1 **NVIDIA GPU**: **NVIDIA Driver Version**: **CUDA...
## Description I created a model which has only one fully connected layer, and want to build it into int8 engine, but it turn out a fp32 engine file. Could...
## Description see also x.log ``` -- [download 49% complete] [ 53%] Linking CXX shared library ../out/libnvinfer_plugin.so /usr/bin/ld: read in flex scanner failed clang-12: error: linker command failed with exit...
when I do BERT inference with trt, I found that it's hard to change the shape with context. Problem： I want to set different shapes for differents inputs, but the...
## Description Use `polygraphy run --trt` and got NaN output. ## Environment **TensorRT Version**: 8.4.1.5 **NVIDIA GPU**: NVIDIA GeForce GTX 1660 SUPER **NVIDIA Driver Version**: 510.85.02 **CUDA Version**: 11.6 **CUDNN...
## Description I did log a issue in Triton server - https://github.com/triton-inference-server/server/issues/4842 and @rmccorm4 suggested to log issue in TensorRT because Polygraphy validation is failed for BERT model. ## Environment...
## Description In my understanding, it is intended to use one of the provided dockerfiles from a release, build it and then run tensor-rt inside. However, I've tried several releases...
