Low gpu utilization

[ArgoHA]

I get around 25% gpu utilization (got that number using gpustats util). That seems to be pretty low. I have one yolov5l model converted to tensorrt in triton server. I have A4000 gpu and several cameras. I don’t get all 25fps, but gpu utilization does not do higher than 25%. Cpu is also used not more than 50% on all cores. How to get more performance? How should I troubleshoot the problem?

[huynhbaobk]

HI @ArgoHA A, so sorry for late response, I think u should try increase the number instances of model in "triton-deploy/models/yolov5/config.pbtxt" (instance_group - count) export new models with larger batch size in "tensorrt-triton-yolov5/tensorrtx/yolov5/yolov5.cpp" -> #define BATCH_SIZE 1 and batch size of inputs in config.pbtxt (max_batch_size)