Inference time questions with warm_up using python_trt.py

[yinhai86924]

Env

• GPU, e.g. TeslaT4 , etc.
• OS, e.g. Centeros7
• Cuda 10.2 version
• TensorRT 6.1.0.8 version

About this repo

• yolov5.5.0, tensorrt-yolov5-5.0

Your problem

after to do warm_up , at the beginning of inference time is 4ms ,after a while, inference time is 7ms, what is problem? I remove the imwrite part of code. [root@ecs-7d42 yolov5]# python3 yolov5_trt.py bingding: data (3, 640, 640) bingding: prob (6001, 1, 1) batch size is 1 warm_up->(640, 640, 3), time->6.83ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->6.80ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->6.80ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->6.81ms warm_up->(640, 640, 3), time->6.82ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.94ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.94ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.96ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.95ms warm_up->(640, 640, 3), time->4.22ms warm_up->(640, 640, 3), time->4.21ms warm_up->(640, 640, 3), time->4.21ms warm_up->(640, 640, 3), time->4.21ms warm_up->(640, 640, 3), time->4.21ms input->['smoke_300/_01_20220214093709711_TIMING.jpg'], time->4.50ms, saving into output/ input->['smoke_300/_01_20220316091712400_TIMING.jpg'], time->4.50ms, saving into output/ input->['smoke_300/10.0.13.101_01_20211205174121085_TIMING.jpg'], time->4.27ms, saving into output/ input->['smoke_300/10.0.118.63_01_20211230093605367_TIMING.jpg'], time->4.32ms, saving into output/ input->['smoke_300/10.0.172.213_01_20220103085728940_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/10.0.190.234_01_20210719161609578_TIMING.jpg'], time->4.25ms, saving into output/ input->['smoke_300/10.0.157.248_01_20211026164913440_TIMING.jpg'], time->4.25ms, saving into output/ input->['smoke_300/10.0.216.211_01_20220522084105331_TIMING.jpg'], time->4.19ms, saving into output/ input->['smoke_300/10.0.79.129_01_20220522191108038_TIMING.jpg'], time->4.34ms, saving into output/ input->['smoke_300/_01_20220214165435297_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/10.0.139.82_01_20210930090406313_TIMING.jpg'], time->4.37ms, saving into output/ input->['smoke_300/10.0.50.100_01_20220217184912253_TIMING.jpg'], time->4.34ms, saving into output/ input->['smoke_300/10.0.168.111_01_20210910175634176_TIMING.jpg'], time->4.26ms, saving into output/ input->['smoke_300/_01_20220127083039606_TIMING.jpg'], time->4.10ms, saving into output/ input->['smoke_300/10.0.118.63_01_20211230092204253_TIMING.jpg'], time->4.28ms, saving into output/ input->['smoke_300/10.0.88.106_01_20210928182912379_TIMING.jpg'], time->4.38ms, saving into output/ input->['smoke_300/10.0.55.163_01_20220518182811872_TIMING.jpg'], time->4.36ms, saving into output/ input->['smoke_300/_01_20211211082412253_TIMING.jpg'], time->4.37ms, saving into output/ input->['smoke_300/10.0.2.134_01_20210925190312321_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/_01_20220414080111270_TIMING.jpg'], time->4.06ms, saving into output/ input->['smoke_300/10.0.157.248_01_20211026172713817_TIMING.jpg'], time->4.37ms, saving into output/ input->['smoke_300/10.0.50.100_01_20220218170416292_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/10.0.104.55_01_20211217155912222_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/10.0.50.100_01_20220217105911154_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/10.0.156.194_01_20220409183913343_TIMING.jpg'], time->4.34ms, saving into output/ input->['smoke_300/10.0.23.125_01_20210720193412602_TIMING.jpg'], time->4.32ms, saving into output/ input->['smoke_300/_01_20220201191006276_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/_01_20220413180944008_TIMING.jpg'], time->4.29ms, saving into output/ input->['smoke_300/_01_20220310083509517_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/_01_20220204150037630_TIMING.jpg'], time->4.27ms, saving into output/ input->['smoke_300/_01_20220408161113168_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/10.0.50.100_01_20220217184411085_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/10.0.126.221_01_20210831183107317_TIMING.jpg'], time->4.25ms, saving into output/ input->['smoke_300/10.0.104.55_01_20211217144911067_TIMING.jpg'], time->4.29ms, saving into output/ input->['smoke_300/_01_20220429172759726_TIMING.jpg'], time->4.29ms, saving into output/ input->['smoke_300/10.0.107.129_01_20211023135509997_TIMING.jpg'], time->4.23ms, saving into output/ input->['smoke_300/10.0.76.139_01_20220118103105927_TIMING.jpg'], time->4.27ms, saving into output/ input->['smoke_300/_01_20211015135618043_TIMING.jpg'], time->4.36ms, saving into output/ input->['smoke_300/10.0.118.63_01_20211230094405925_TIMING.jpg'], time->4.27ms, saving into output/ input->['smoke_300/_01_20220203180837854_TIMING.jpg'], time->4.07ms, saving into output/ input->['smoke_300/10.0.104.55_01_20211217151412823_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/_01_20220127081839751_TIMING.jpg'], time->4.10ms, saving into output/ input->['smoke_300/10.0.137.231_01_20211205141720607_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/_01_20220429173559956_TIMING.jpg'], time->4.27ms, saving into output/ input->['smoke_300/_01_20220316091912400_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/10.0.107.129_01_20211024184120629_TIMING.jpg'], time->4.07ms, saving into output/ input->['smoke_300/_01_20211130103854941_TIMING.jpg'], time->4.38ms, saving into output/ input->['smoke_300/10.0.139.126_01_20220103144909339_TIMING.jpg'], time->4.35ms, saving into output/ input->['smoke_300/10.0.191.141_01_20211210173408489_TIMING.jpg'], time->4.37ms, saving into output/ input->['smoke_300/_01_20220111095556956_TIMING.jpg'], time->4.34ms, saving into output/ input->['smoke_300/10.0.104.55_01_20211217164412841_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/_01_20220331185619771_TIMING.jpg'], time->4.30ms, saving into output/ input->['smoke_300/_01_20220414151110592_TIMING.jpg'], time->4.06ms, saving into output/ input->['smoke_300/10.0.221.167_01_20210921183514984_TIMING.jpg'], time->4.30ms, saving into output/ input->['smoke_300/10.0.157.248_01_20211026155313939_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/_01_20220209154037289_TIMING.jpg'], time->4.36ms, saving into output/ input->['smoke_300/10.0.118.63_01_20211230102405583_TIMING.jpg'], time->4.31ms, saving into output/ input->['smoke_300/_01_20220127082839729_TIMING.jpg'], time->4.12ms, saving into output/ input->['smoke_300/_01_20220120092219937_TIMING.jpg'], time->4.32ms, saving into output/ input->['smoke_300/10.0.2.134_01_20210926183922047_TIMING.jpg'], time->4.38ms, saving into output/ input->['smoke_300/10.0.148.85_01_20211201153913669_TIMING.jpg'], time->4.24ms, saving into output/ input->['smoke_300/_01_20220218153509474_TIMING.jpg'], time->4.32ms, saving into output/ input->['smoke_300/10.0.79.129_01_20220522190108987_TIMING.jpg'], time->4.34ms, saving into output/ input->['smoke_300/_01_20220217112111372_TIMING.jpg'], time->4.20ms, saving into output/ input->['smoke_300/_01_20220311084910265_TIMING.jpg'], time->4.18ms, saving into output/ input->['smoke_300/10.0.67.83_01_20211024160911667_TIMING.jpg'], time->7.21ms, saving into output/ input->['smoke_300/10.0.172.213_01_20220103084528746_TIMING.jpg'], time->7.27ms, saving into output/ input->['smoke_300/_01_20220302082708870_TIMING.jpg'], time->6.82ms, saving into output/ input->['smoke_300/_01_20220212082255642_TIMING.jpg'], time->7.27ms, saving into output/ input->['smoke_300/10.0.50.100_01_20220217182911146_TIMING.jpg'], time->7.26ms, saving into output/ input->['smoke_300/10.0.221.167_01_20210921191914839_TIMING.jpg'], time->7.08ms, saving into output/ input->['smoke_300/10.0.44.152_01_20211221181109060_TIMING.jpg'], time->6.98ms, saving into output/ input->['smoke_300/_01_20220404084836211_TIMING.jpg'], time->7.17ms, saving into output/ input->['smoke_300/_01_20220329083311589_TIMING.jpg'], time->7.16ms, saving into output/ input->['smoke_300/10.0.137.231_01_20211205150320545_TIMING.jpg'], time->7.32ms, saving into output/ input->['smoke_300/_01_20220122185040779_TIMING.jpg'], time->7.27ms, saving into output/ input->['smoke_300/10.0.35.188_01_20220328105106459_TIMING.jpg'], time->6.93ms, saving into output/

[wang-xinyu]

Maybe it's related to the GPU power mode.

[yinhai86924]

Maybe it's related to the GPU power mode.

I use Tesla T4, How to set the GPU power mode or select what mode of GPU compute. Thanks for your help!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.