Kevin Chen comments

Results 107 comments of


                                            Kevin Chen

how to create TensorRT Run time for two inputs model and one output

Closing at it seems to be resolved.

Engine build failure "cuda misaligned address" of TensorRT 10.1 when running fp16 group normalization with particular value of `num_groups`

Using TRT 10.8 I'm unable to repro the issue, all test cases pass. @haijieg can you confirm that 10.8 works for you?

Could not find any implementation for node /backbone/layers.0.1.0/attn/attn/MatMul for SegFormer onnx model for int8 precision

Closing due to issue being reported on TRT version that we no longer support. Please try with the latest TRT version to see if you still have the issue.

Why tensorrt’s performance is poor after adding custom op

All plugins (written by NVIDIA or not) break graph fusions in TensorRT. User written plugins are recommended for operators that are not supported by TensorRT or if the plugin implements...

set fp32 for MatMul failure of TensorRT 8.6.1.1 when converting onnx to trt with version 8.6.1.1

You can use the builder flag kSTRONGLY_TYPED to force TensorRT to respect the casts. See https://github.com/NVIDIA/TensorRT/blob/release/10.8/include/NvInfer.h#L10299 for more info.

How to achieve throughput in trtexec with python api?

`trtexec` is mainly a pure performance measuring tool, and uses a few tricks to increase number of queries, including the fact that it enqueues two batches at once, and doesn't...

AttributeError: 'tensorrt_bindings.tensorrt.ICudaEngine' object has no attribute 'get_binding_index'

Are you still having this issue? Are you able to share the model? Does inference work using trtexec?