TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

tensorrt test time is not stable

Open jiaqizhang123-stack opened this issue 2 years ago • 3 comments

Hello, when I use c++ to infer the engine file, when the nvidia driver version is a new version, the inference time is not stable(16-200ms) image

But inference time is stable（16ms） when nvidia driver version is old. For example, nvGameS.dll is 27

I also tested different cuda version and trt version, I want to ask what is the reason for this, thanks for your answer.

jiaqizhang123-stack avatar Sep 01 '22 11:09 jiaqizhang123-stack

could you set your GPU to Performance mode?

nvpohanh avatar Sep 01 '22 11:09 nvpohanh

Where should this be modified?

jiaqizhang123-stack avatar Sep 01 '22 11:09 jiaqizhang123-stack

By modifying the parameters of the nvidia control panel, the test time is still unstable. What is the reason for this?

jiaqizhang123-stack avatar Sep 02 '22 05:09 jiaqizhang123-stack

Could be a driver bug...

nvpohanh avatar Dec 02 '22 09:12 nvpohanh

Closing since this seems not TRT bug, also for Windows I believe the OS and other app also try to use the GPU resource. Might need enable Windows TCC mode to make the perf stable.

ttyio avatar Jan 17 '23 02:01 ttyio