Thank you for excllent work. How about TRT batch inference?

[tungdq212]

trafficstars

Thank you for excllent work.

Detection models now can be exported to TRT engine with batch size > 1 - inference code doesn't support it yet, though now they could be used in Triton Inference Server without issues.

Is there any plan for this? Or how can I implement batch inference myself?

[SthPhoenix]

Hi! Batch inference is already supported for all recognition models and for SCRFD and YOLOv5 family detection models.