Improve pico-controls on third-party browsers such as Wolvic

[Elettrotecnica]

It appears that Pico Browser applies some adjustments on controllers to somewhat mimic an Oculus controller. On a neutral browser such as Wolvic, these adjustments are not applied. The end result is that turning and gestures do not work on Wolvic when using hand-controls.

The following changes make the mappings on a pico controller more similar to those on the oculus controller, which seems to improve the situation: gestures and turning works.

TODO: its seems that Pico Browser also applies an offset to the controller model, again similar to that we apply on Quest controllers. I have not addressed this yet.

[Elettrotecnica]

At https://fluff-peat-roundworm.glitch.me you can see a live demo of the issue.

Using the stock Pico browser, the behavior is correct: hands produce gestures and we can turn around using the thumbstick.

With Wolvic, hands will be shown, but no gestures and no turning.

I can produce the bug on a Pico4 device. I can not tell older devices also behave the same.

[mrxz]

Using the stock Pico browser, the behavior is correct: hands produce gestures and we can turn around using the thumbstick.

Perhaps I'm missing something obvious but there doesn't seem to be any turning logic in the demo you shared, just the hand-controls which should only provide the basic hand model and gestures.

In any case, A-Frame should not adjust code on a per browser level, only on a per controller level. WebXR provides profiles for each controller that specify the button and axes mappings. If a browser's WebXR implementation indicates a profile but behaves differently, that should be fixed in the browser, not A-Frame.

That being said, the changes do look good as they seem to fix mistakes in the original implementation. The squeeze -> grip is an interesting change, as the WebXR Device API spec and WebXR input profiles tend to use squeeze, but grip is used by A-Frame in several locations. This should really be streamlined, and while I do prefer squeeze for consistency with the spec and profiles, grip is already relied on by various A-Frame components and apps... Either way, that is outside the scope of this PR.

The puzzling part is how this change could "fix" Wolvic on Pico, while continue working on the Pico Browser. My guess is that the Pico Browser sends additional profile ids, which results in A-Frame not actually using pico-controls, but a different implementation (possible oculus-touch-controls). Could you test/confirm this?

[Elettrotecnica]

Perhaps I'm missing something obvious but there doesn't seem to be any turning logic in the demo you shared, just the hand-controls which should only provide the basic hand model and gestures.

~hand-controls also introduce turning via the thumbstick. In my demo, using wolvic on Pico4 you won't be able to turn.~ My bad, you are right: hand-controls won't make you turn, it's the blink-controls component on my website that does it. However, even with blink-component turning will not work on pico4 without changing the axes. I have updated my demo.

The puzzling part is how this change could "fix" Wolvic on Pico, while continue working on the Pico Browser. My guess is that the Pico Browser sends additional profile ids, which results in A-Frame not actually using pico-controls, but a different implementation (possible oculus-touch-controls). Could you test/confirm this?

Will do!

[Elettrotecnica]

Yeah, I have put some log statements in the event handlers for the oculus-touch-controls component and when I enter my site with the Pico Browser they will fire.

It seems that the stock Pico browser is really pretending to use quest controllers... This was also my suspicion seeing how the controller offset is different on stock and wolvic: on Wolvic, the hands will appear in a tilted "wrong" position, while on stock they will be fine.

IMO, the Pico browser is "cheating", but is doing it "correctly", meaning that we could probably copy paste a big chunk of the oculus code into the pico component and this should work fine in wolvic... I am not sure though if this can be said for all Pico devices or only for the pico4...

What would be a good way to address this in your opinion?

[mrxz]

Actually, it might still be a bug in A-Frame as the controller code is quite iffy in places. The oculus-touch-controls activates as soon as there is any oculus-* or meta-* profile id in the list. Once activated it can block other controls components from being used. It's possible that Pico includes the Oculus touch profile id, but not as its first one.

Could you log the list of profile ids on the controller?

document.getElementById('leftHand').components['tracked-controls-webxr'].controller.profiles

If that is indeed the case, we'll have to fix that first, as logically Pico devices should use the pico-controls. I do have a Pico 3 lying around somewhere, so I'll be able to test that sometime.

[dmarcos]

@Elettrotecnica Can you share the complete list of profiles as @mrxz described? Thanks

There are Pico profiles submitted by the team: https://github.com/immersive-web/webxr-input-profiles/tree/main/packages/registry/profiles/pico

Browsers on the platform (Wolvic and built-in) should report those and not the oculus controls because model and potentially button mapping will be incorrect.

@song-fangzhen What are the current browser plans to expose the Pico profiles and remove the quest ones? I'll make sure that Pico controllers are well supported in all the built-in components (hand-controls, laser-controls...) Thanks so much for the hard work on Pico. Awesome device.

[Elettrotecnica]

Dear all, these are the controller profiles reported on the two browsers for Pico4:

Mozilla Fennec (121.0.1) A81E0, buildID 20240125

[ "pico-4", "generic-trigger-squeeze-thumbstick" ]

com.pico.browser.overseas (105.0.5195.68)

[ "oculus-touch-v2", "pico-phoenix", "pico-neo3", "pico-neo2", "oculus-touch", "generic-trigger-squeeze-thumbstick" ]

[dmarcos]

Thanks so much. Both browsers should be consistent. Interesting that the built-in browser doesn't even report a Pico 4 profile on the device. I pretty much prefer the profiles to be fixed on the browser side. Wolvic also looks incomplete. It should also return a generic profile. I also don't think the built-in browser should report Pico2, Pico3...

[Elettrotecnica]

I agree with you @dmarcos that profiles on the stock browser are weird, although the end result is that controllers work. How do you suggest we could proceed to improve the situation on Wolvic? IMO, because Wolvic reports the correct controller, we should improve the pico-controller component to better support pico-4.

[dmarcos]

@Elettrotecnica Yes, we should support pico-4 profile. Wolvic should then work.

For the built-in browser I think we should ask the team for a fix. It's not following the standard.

[dmarcos]

I see we already have support for the Pico 4

[Elettrotecnica]

I see we already have support for the Pico 4

The pico-controls component mentions pico-4, e.g. it hardcodes the GAMEPAD_ID variable (see https://github.com/aframevr/aframe/blob/66baa9c8128caf9199e8a773ca4d637a25947d06/src/components/pico-controls.js#L12)

There is probably some massaging needed to improve it. I am not sure if this will affect other pico models, or what the current status for those models is atm.

I believe something like it has been done for oculus-touch-controlls may be needed, e.g. support for multiple profiles.

[dmarcos]

We should definitely support the pico4 profile properly. Not sure if it make sense doing the same as oculus touch. Different touch versions have exactly the same number of buttons and matching indices. Not sure if it's the case for all pico devices. Also wonder how many Pico headsets older than 4 are active out there. Complexity might not be worth.

For now we can support the pico4 and expand later if needed.

[dmarcos]

Also It seems that Wolvic is following the standard closer than the built-in browser. Wolvic at least returns the correct profile for the device (pico-4).

[Elettrotecnica]

Ok, then if it is fine for you, given that I have a pico4 handy, I will expand this PR to ensure that on Wolvic, hand-controls works as expected.

Gestures and turning should already be fine with the current changes. The only thing missing as far as I understand is applying the right offset to the controller so that it is rotated correctly.

[dmarcos]

I would try pico-controls by itself first and make sure that the correct model is loaded and matches the physical one

[Elettrotecnica]

Dear all,

I have added a commit to address the different rotation offset on hand-controls for the Pico4 on Wolvic.

To test that the change behaves as expected I have prepared a couple of Glitch demos. In all of them I have made so that one can switch from latest official Aframe release to a custom build containing changes in this PR.

• https://fluff-peat-roundworm.glitch.me/ - hand-controls example
• https://destiny-spangled-botany.glitch.me/ - oculus-touch-controls example
• https://silent-prickle-ship.glitch.me/ - pico-controls example

Note that according to https://developer.picoxr.com/document/web/development-platform/#2451b38a the Pico Browser will transparently inject an own model instead of the oculus controller, so the pico4 controller we see on the stock browser when using oculus-touch-controls is not the one coming from aframe.

Notice also that because the Pico stock browser does not report to use pico4 controllers, the pico-controls example will not work. This is expected as things stand now.

All the best

[dmarcos]

@Elettrotecnica If we move logic from pico-controls to hand-controls this is close

[Elettrotecnica]

Should be able to check it tomorrow :-)

[Elettrotecnica]

I have moved the offset logics in hand-controls

[Elettrotecnica]

@dmarcos sorry for the bump, is there anything else I should do here? Would be nice to have this merged so my experience works on Wolvic with Pico4 as well :-)

[dmarcos]

Thanks for the patience. Happy to have Pico support improved

[Elettrotecnica]

Thanks to you!