TensorRT

TensorRT copied to clipboard

ONNX Runtime convert to TensorRT: fp32 & fp16 mode success but int8 mode fail

1

## Description ### For fp16 (trt_log_level: trt.Logger.INFO) 2022-09-08 17:36:02,762 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess 2022-09-08 17:36:03,426 - mmdeploy - INFO - Successfully loaded tensorrt plugins...

zml24

Multiple streams in parallel

2

Hello, when I use stream to perform parallel inference on trt, and use nvvp to visualize the time of trt, it can be found that the memory copy and the...

jiaqizhang123-stack

about Internal Error (downsample_ratio: network input that is shape tensor must have type Int32)

5

Description I have an model and my inputs have a downsample ratio that must be Float32. when i onnx2trt , I have onnx parsing falid,What should I do ## Environment...

zhanghongyong123456

The mnist sample test give incorrect output

2

## Description I install the cuda and cudnn with local deb package following the nvidia document, and install tensorrt with tar file. nvidia-smi, nvcc -V, and cudnn test sample is...

yuanjiechen

GeluPlugin corrupted double-linked list

1

## Description When I use TensorRT API and GeluPlugin to build my network, there has a bug: corrupted double-linked list. ## Environment **TensorRT Version**: 8.4.0.6 **CUDA Version**: 10.2 **CUDNN Version**:...

oreo-lp

onnx and trt infers different results in some shapes

2

## Description Hi, i found the results is different in some special shapes when using mmdet proj, the result is 10 pixels shifting in x direction. the solution is i...

congjianting

Dockerfile ubuntu 18.04 build failed

2

## Description [ubuntu-18.04.Dockerfile](https://github.com/NVIDIA/TensorRT/blob/main/docker/ubuntu-18.04.Dockerfile) fails to build, probably something to do with `apt-get install -y software-properties-common` installing `python 3.6` by default, then the subsequent steps require `python 3.10`. ## Environment **TensorRT...

xoiga123

Incorrect calculations in GridAnchorGenerator, gridAnchorLayer for rectangular inputs

2

## Description If the input layer of an object detector is not square, libnvinfer_plugin does not produce the correct bounding boxes. An incomplete fix was committed to master with https://github.com/NVIDIA/TensorRT/pull/679...

marcbelmont

What is allocation memory and why the engine need to use so large activation memory?

1

 As you can see in the picture，the engine generated by tensorrt uses such a huge amount of activation memory. I wonder how I can decrease this usage. The tensorrt...

frankxyy

Wangzheee