Does it support dynamic batch inference?

[wangjingg]

When I try onnx->tensorrt and use dynamic batch, the tensorrt output is all [0,0,0,0,.....]

[maggiez0138]

When I try onnx->tensorrt and use dynamic batch, the tensorrt output is all [0,0,0,0,.....]

Tried dynamic batch, failed. Both onnx model and tensorRT engine can not output the expected output. Maybe this issue has some relationship with the implementation of Swin Transformer. Not sure.

[wangjingg]

When I cancel the softmax, in pytorch->onnx(dynamic batch), It's work, you can try it,

[maggiez0138]

When I cancel the softmax, in pytorch->onnx(dynamic batch), It's work, you can try it,

Thanks for your info, will do the attempt.

[fatemebafghi]

Hello there. I have the exact same problem with dynamic inputs, but I could not find a way to cancell softmax during converting pytorch model to onnx. Can you help me through this?

[maggiez0138]

Hello there. I have the exact same problem with dynamic inputs, but I could not find a way to cancell softmax during converting pytorch model to onnx. Can you help me through this?

Updated the repo. Now dynamic input is supported.