TensorRT

TensorRT copied to clipboard

after tensorrt init finished, my own stream invalid

3

I have a TensorRT ver 8.2.3.0 and a nvJPEG ver 11.6.2.8 Tesla 4 GPU with driver ver 510.47.03 ubuntu & x86_64 here's my situation: I made a dynamic library which...

neiblegy

How to build arm Architecture TensorRT OSS v6.0.1

4

## Description I want to build arm Architecture TensorRT OSS v6.0.1 I used the minimum version of cuda and cudnn that supports arm ``` cuda_11.0.2_450.51.05_linux_sbsa.run cudnn-11.2-linux-aarch64sbsa-v8.1.0.77.tgz ``` but according to...

jamdodot

Mixed precision(FP16+FP32) engine's memory size

10

## Description When I use the mixed precision(fp16+fp32) engine, I find that the memory size of the mixed precision engine is very close to the fp32 engine. Here are ops...

oreo-lp

Conv2d stride=2 accuracy mismatch between PyTorch and TensorRT

7

## Description Pytorch Conv2d operation with `stride=2` after ONNX conversion and TensorRT inference shows accuracy mismatch against PyTorch inference. This is not seen for `stride=1`. The conv2D signature is `nn.Conv2d(64,...

schegde

mAP drops a lot when Infer a INT8 quantized ONNX model.

1

## Description Hi, I have a quantized Yolov5s ONNX model; When I use ONNX runtime to infer this model, I got the mAP of 36.8; But when I use C++...

yunyaoXYY

Question about trtexec log

1

I use trtexec to convert my onnx model to int8 engine. But I find some log like 'Dynamic range would not be set for tensor (Unnamed Layer* 36) [Constant]_output for...

pycoco

Question: trtexec hardware during compilation vs production?

1

I am using `tensorRT` together with [Triton](https://github.com/triton-inference-server/server). Inorder to avoid a long runing start up time when we deploy to edge nodes I would like to do the tensorRT conversion...

NikeNano

How to get the profile by python?

1

## Description I can get the profile by trtexce: `./trtexec --loadEngine=debug_fp16.trt --dumpProfile --shapes=input:1x3x512x512 --exportProfile=debug_profile ` How can I get the debug_profile by python when I convert onnx to trt engine...

oreo-lp

Getting “failed to import metagraph” when running the TRT saved_model converter workflow for TF 1.15

2

I have trained a NER model with tensorflow1.15 and save the model in pb file . when I want to convert it to a new pb with tensorRT. Error log...

liutao053877

Should I use pytorch-quantization or not?

5

## Description I have trained a model by using pytorch and export an onnx model. Now, I want to run it on TensorRT with fp16. Should I use the pytorch-quantization...

jackgao0323