How to use ARCamera (Vuforia) with OpenCVForUnity?

[bumchikiboss]

I want to implement Vuforia's ARCamera with TextRecognitionCRNNWebCamExample. It uses default main camera and script - WebCamTextureToMatHelper. It throws timeout error when using AR camera. What do I need to do in order to use AR Camera with Text Recognition. Sorry I'm quite novice in this.

[EnoxSoftware]

WebCamTextureToMatHelper cannot be used because WebCamTexture is not used in Vuforia's ARCamera. This repository is an example of how to convert from Vuforia's ARCamera to OpenCV's Mat. https://github.com/EnoxSoftware/VuforiaWithOpenCVForUnityExample/

[bumchikiboss]

Ok so I have used the PostRenderToMatExample.cs from the repo you mentioned and put elements from TextRecognitionCRNNWebCamExample script in it. Its working with ARCamera but the thing is that its very slow. the framerate I'm getting is like 5-6 fps. is there any solution for this...maybe I can tinker with some values or something.

[EnoxSoftware]

There is a way to do an asynchronous conversion from Texture to Mat using the Rendering.AsyncGPUReadback class. Asynchronous conversion may improve FPS. https://docs.unity3d.com/2021.2/Documentation/ScriptReference/Rendering.AsyncGPUReadback.html

The following code is an example of an asynchronous conversion using the Rendering.AsyncGPUReadback class in OnPostRender method. https://github.com/EnoxSoftware/OpenCVForUnity/blob/2.5.7/Assets/OpenCVForUnity/Examples/MainModules/videoio/VideoWriterAsyncExample/VideoWriterAsyncExample.cs#L188-L271