TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
In the EfficientNMS Plugin Introduction, "After sorting, the highest 4096 scores are processed by the EfficientNMS CUDA kernel. " Can this "4096" be configured? In my understanding , the 4096...
## Description Set_binding_shape returns False ``` import os import sys sys.path.append(os.path.abspath(os.path.dirname(os.path.dirname(__file__)))) import os.path import numpy as np from multiprocessing import cpu_count import cv2 import shutil from PIL import Image from...
There is int type data in my inputs for indexing other data, how can I set the int type data as input, with set_input?
## Description [08/04/2022-17:24:38] [TRT] [V] *************** Autotuning Reformat: Float(E8,E8,E7,E6,(* 108 E1),108,1) where E0=(- 108 (- (# 3 (SHAPE input_1)) (* 108 (CEIL_DIV (+ (# 3 (SHAPE input_1)) -107) 108)))) E1=(CEIL_DIV...
Change the input image size of `ONNX` model from `1 * 3 * 1024 * 1024` to `1 * 3 * 2000 * 2000`, get an error when `converting ONNX...
In the only example provided in the toolkit, it loaded the PTQ calibrated weights and did the QAT based on it. There isn't a standalone QAT example without PTQ. I...
## Description I export onnx model and build engine with FP16 mode successfully, but the issue occurs when building engine with INT8 calibration mode. The error: ``` ... [06/30/2022-17:48:48] [TRT]...
## Description It is caused by a bug from pytorch, as shown in the issus [ "**torch.fake_quantize_per_tensor_affine will affect the results of the model if the input memory is not...
## Description Follow https://github.com/NVIDIA/TensorRT/blob/main/demo/HuggingFace/notebooks/t5.ipynb to build the tensorRT example for Bart model. Get warning in `bart_trt_decoder = BARTTRTDecoder(bart_trt_decoder_engine, metadata, tfm_config)` Cannot find binding of given name: past_key_values.0.decoder.key and error in...
Hey, ## Background Opencv introduce Compute graph, which every Opencv operation can be describe as graph op code. They took it further and, introduces the ability to use inference on...
