Using a photo set to test frame processor plugins❓

[Easy47]

Question

How to test the accuracy of frames processor?

What I tried

I would like to be able to measure the accuracy of frame processor plugins QR code reader and OCR.

To do this I would like to be able to give to the two frame processor plugins a photo set and observe the results for each photo of the set and measure the accuracy ? To sum up, I would like to be able to test frame processor plugins alone with a photo set.

Ideas ? Thanks in advance :)

VisionCamera Version

latest

Additional information

• [ ] I am using Expo
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar questions in the issues page as well as in the discussions page and found none.

[mrousavy]

Well you can just count the successful reads yourself:

const successfulReads = useSharedValue(0)

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  
  const textInImage = scanTextInImageOCR(frame)
  if (textInImage.length > 0)
    successfulReads.value += 1
}, [])

[Easy47]

Thank you for your answer. However, this is not possible: To be able to measure the impact of image pre-processing (for example), we need to ALWAYS have the same inputs (exactly the same). This is the main problem. We really need a way to be able to control the input flow exactly.

[wojteklembryk]

Hi, I have similar use case where I'd like to pass an image to frameProcessor. Is it possible to map Image to Frame?

[Easy47]

Did you find something ?

[EmielStoelinga]

@Easy47 @wojteklembryk any updates on this? I am facing the same problem in which I want to be able to test the plugin using a static image.

[Easy47]

@Easy47 @wojteklembryk any updates on this? I am facing the same problem in which I want to be able to test the plugin using a static image.

I made a frame processor plugin in which I load an image. I did not succeed in directly returning the frame so I test in the same plugin

[wojteklembryk]

@Easy47 @wojteklembryk any updates on this? I am facing the same problem in which I want to be able to test the plugin using a static image.

I made a frame processor plugin in which I load an image. I did not succeed in directly returning the frame so I test in the same plugin

I changed approach and instead of reusing frame processor I extracted it's logic into separate component. My case was Google MLKit based so I use image picker that's pass data to MLKit bridge and then apply shared frame processor logic.

[mrousavy]

Closing as this is a stale issue - this might have been fixed with the full rewrite in VisionCamera V3 (🥳) - if not, please create a new issue.