wasi-nn use a batch of image as input

Hi guys, is there any example that can convert multiple image to one tensor with dimension [batch_size, channel, witdth, height]? I am hoping this may be more efficient towards to real inference scenarios.

Jan 17 '22 06:01 ln23415

I believe this may be possible with the OpenVINO backend in Wasmtime, but it could bear some investigation. The wasi-nn bindings repository has Rust and AssemblyScript examples of running an inference with a single image and you could adapt those to batch several images into one request. I think, but have not checked, that the first dimension in those examples can be used to indicate that the tensor contains multiple images (but now that I'm looking at the docs, I'm wondering if OpenVINO also needs some configuration). If you are interested in discussing further, I am usually online on Zulip if you want to chat.

Jan 21 '22 05:01 abrown

thanks! I'm checking the soluation as well, I see in openVINO python or C APIs, there is a attribute to open inferencer's batch mode, maybe find out where to open this attribute in IR code could be useful.

Feb 04 '22 00:02 ln23415

@ln23415, did you figure out how to pass multiple tensors in through the wasi-nn API?

Jun 13 '22 16:06 abrown