TensorRT

TensorRT copied to clipboard

the result on TensorRT10.3 is difference from TensorRT10.8 and is wrong

2

The results of tensorRT10.3 are different from those of TensorRT 10.8. The results of 10.8 are as expected, so the results of 10.3 are wrong. I want to use 10.3...

huihui6666

Tensorrt optimization shows unexpected results

5

Hi, i try to create tensorrt engine from an onnx model. I tried a few things and here are the inference latencies. Why is 3. and 4. performing worse than...

geraldstanje

After upgrading from 8.6 to 10.8 or 10.9, tensorrt's results are inconsistent with onnxrt

11

## Description After upgrading tensorrt to 10.8, the model accuracy decreased. After setting all nodes of the model to output, the model accuracy was aligned. It was suspected that the...

2730gf

Conversion to TRT failure of TensorRT 8.6.1.6 when converting CO-DETR model on GPU RTX 4090

4

## Description I tried to convert model [CO-DETR](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/CODETR.md) to TRT, but it fails with error below ```bash [12/12/2024-02:17:38] [E] Error[10]: Could not find any implementation for node {ForeignNode[/0/Cast_3.../0/backbone/Reshape_3 + /0/backbone/Transpose_3]}....

edwardnguyen1705

Assertion failed of TensorRT 10.9 when export nvidia-embed-v2 ONNX Model to TensorRT (polygraphy and trtexec)

2

--- name: Report a TensorRT issue about: Failed to export ONNX Model (Transformer) to TensorRT title: 'Assertion failed of TensorRT 10.9 when export ONNX Model to TensorRT (polygraphy and trtexec)'...

ducknificient

Set layer precision still doesn't take effect in TensorRT 8.6.1.

21

## Description As I had reflected in this [Skipping tactic 0x0000000000000000 due to Myelin error" degrade performance.](https://github.com/NVIDIA/TensorRT/issues/2838)，set layer precision may failed in TensorRT 8.4.3 due to the ConstShuffleFusion. In these...

YouSenRong

Failure of TensorRT 10.7 to eliminate concatenation with upstream custom layer

8

## Description It seems that TensorRT cannot eliminate a concatenation layer if there is an upstream custom layer. In a simple model that uses all standard operators, TensorRT engine building...

jchia

Explicit quantization is slower than implicit quantization and produces invalid results

2

## Description Since implicit quantization is deprecated, I started migrating my model pipeline to explicit quantization. However, I encountered some issues: 1. Different behaviour with concat: With implicit quantization the...

itmo153277

How are qparams (scale and zero_point) determined after fusing Conv and BN layers?

1

During quantization (using pytorch_quantization), the qparams (scale and zero_point) of old Conv is computed using Calibrator. However, when the Conv and Batch Normalization (BN) layers are fused, the weights and...

gef1998

Given an engine file, how to know what GPU model it is generated on?

8

When I use `trtexec` and I mix TensorRT engine plan files across different GPU models, I can get a warning: ``` Using an engine plan file across different models of...

yangdong02