Low perfomance with batch size > 1

[elyha7]

I am getting about 10 times less fps on rk3568 chip, inferencing model with batch size more than one. Is it some chip peculiarities or i am doing something wrong? Here are configurations and model perfomance screenshots. Batch size 1: Batch size 2:

[blueskywwc]

Hello, I would like to ask on the board how to set batch>1 inference. Thank you

[githubabin]

hello， I set rknn_batch_size>1 during the phase building 3568 rknn model with rknn-toolkit2 1.3.0. but the rknn model is still batch = 1, when use it to inference.

[blueskywwc]

thanks，View the knn document, some operators in rknn only support rknn_batch_size=1, such as pooling and activating 。Therefore, the rknn_batch_size>1 setting does not work。

[datpham270198]

Hi. I converted my model with batch_size > 1 on simulator toolkit2 and also infer on PC. So can someone tell me, how to inference with batch_size > 1 on rknn board. I get stuck there!!! Many thanks

[medvedev]

thanks，View the knn document, some operators in rknn only support rknn_batch_size=1, such as pooling and activating 。Therefore, the rknn_batch_size>1 setting does not work。

@blueskywwc do you know if this is still true for toolkit v2.0.0(beta) and if so, if these restrictions are documented anywhere?