Zero Zeng
Zero Zeng
How do you measure TRT perf. could you please try trtexec? it's a binary come with TRT package for perf measurement, a typical usage would be like `trtexec --onnx=model.onnx --fp16...
> [01/03/2024-23:26:12] [W] Dynamic dimensions required for input: img, but no shapes were provided. Automatically overriding shape to: 1x3x1x1 > [01/03/2024-23:26:12] [W] Dynamic dimensions required for input: input_ids, but no...
> Finally, it was successfully converted into an engine file, but when inferring the engine, the accuracy still cannot be aligned. It kind of like caused by pre-processing or post-processing.
not likely, you can comfirm this with polygraphy, usage like `polygraphy run model.onnx --trt --onnxrt`
How big is the diff? we don't guarantee bit-wise alignment for TRT and other frameworks.
Could you please explain what is your use case and what do you want to do? Sorry I don't quite get the report. Thanks!
+ @nvpohanh @oxana-nvidia for viz
TRT 9 is a limited release so we didn't put it in dev zone. How ever it can be downloaded from below links. https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.linux.x86_64-gnu.cuda-11.8.tar.gz https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.linux.x86_64-gnu.cuda-12.2.tar.gz https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.ubuntu-22.04.aarch64-gnu.cuda-12.2.tar.gz