javacv icon indicating copy to clipboard operation
javacv copied to clipboard

Passing OpenCV Mat as Tensorflow input

Open barrypitman opened this issue 1 year ago • 2 comments

Firstly, thanks for a great project, it's fun to work with!

I have an org.bytedeco.opencv.opencv_core.Mat object that I have prepared into the correct shape, format & scaling to pass to a tensorflow lite model. I'm having trouble populating the model input tensor correctly.

Mat image = ...; // 300 x 300 x 3 channels, depth=32
FloatPointer input = interpreter.typed_input_tensor_float(0);

I've got it working by iterating over the rows and cols of the image and copying the image pixels one by one into a float[] and calling input.put() on the float array. However, that seems inefficient, and I was hoping that I could do something like input.put(image), but that isn't working, or at least the model outputs aren't correct.

Could you give me any pointers (!) on how to efficiently copy/pass the Mat as a tensorflow input?

Thanks in advance.

barrypitman avatar Apr 23 '24 12:04 barrypitman

Indexers are efficient: https://bytedeco.org/news/2014/12/23/third-release/

saudet avatar Apr 23 '24 12:04 saudet

So something like this? input.put((float[]) image.createIndexer(false).array());

I was wondering whether it would be better to pass a pointer to the Mat (and avoid loading the floats into java memory at all), but that didn't seem to work.

barrypitman avatar Apr 23 '24 12:04 barrypitman