Dynamic Batch Inference

[UygarUsta99]

I have exported my yolov7 model with dynamic axes as tensorrt.Engine runs fine on single batch but I want to be able to make inference with multiple images.How can I achive this ? How should I feed my images to the model.predict method ? I have tried using model.predict(np.array([im.copy(),im.copy()])) but I get errors. Any help would be much appreciated.

[jiangjiajun]

Hi, @UygarUsta99 We are working on batch inference this week, I think it will supported in next week. Once the feature is updated, we will comment here to let you know. :)

[heliqi]

Temporary solution:

im_list = [im, im, im]
for i in range(im_list):
    model.predict(i)

or

im_numpy = np.array([im, im])
for  i range(len(im_numpy)):
    model.predict(im_numpy[i])