Update on access to camera image pixels?

[pavan4]

Hello,

I followed @blairmacintyre 's discussion closely regarding the access the camera image pixels when using WebXR. I am not looped in on the latest discussion happening around the spec development and hence my question here.

Currently, I am able to access the camera pose via the webXR API and tried to access the pixel data using various methods (some listed here) Understandably all access to the pixel data is blocked. Is there any way to access the image data?

I am exploring the usage of WebXR and wanted to ask if there has been any update on access to camera image. I also want to document a usecase - At Streem we use AR for customer support. A remote professional can dial in via AR and support a customer remotely. We use CV extensively and connect the video via webRTC. Hence, access to the image data is highly useful with appropriate user permission prompts obviously.

[boehm-e]

Any update, i'm interested too

[useronym]

Same

[TheYass1n]

any updates on this

[pavan4]

My update after research is that this feature is under Dev trials on chrome for Android under WebXR. More info. can be found here -

• WebXR
• Android API
• Sample code

Hope this helps someone

[klausw]

More specifically, this is addressed by the WebXR Raw Camera Access Module, see the draft spec and explainer. For sample code, check out this barebones example and using OpenCV ArUco marker tracking.

The feature is enabled by default in Chrome Canary >= 107.0.5304.0. For current stable Chrome, you need to enable "WebXR Incubations" in chrome://flags .

[TheYass1n]

This is awesome, thanks for the info.

[marcofugaro]

@klausw thanks for the example, do you know why it isn't included in the main webxr-samples website?

https://immersive-web.github.io/webxr-samples/