Raycast + Instancing in A-Frame?

[diarmidmackenzie]

Description:

I maintain an instancing component for A-Frame. https://github.com/diarmidmackenzie/instanced-mesh

I'd like to add raycasting support, as per this Three.js example: https://threejs.org/examples/?q=instancing#webgl_instancing_raycast

This will either require:

1. Updates to A-Frame raycaster component or
2. A new parallel raycaster component for use when using instanced meshes.

I'd much prefer 1., and I think the updates required to A-Frame are fairly modest in scope, and won't have any negative impacts when not using instanced meshes.

Purpose of this issue is to sound out whether you'd be likely to accept a PR for the necessary changes.

The issue

When raycasting against an instanced mesh, there is additional data provided on the intersectObject indicating which instanceId on the instanced mesh is intersected with. This can be used by an application to determine which instance in the instanced Mesh has been targeted, so that it can provide appropriate handling. https://threejs.org/docs/#api/en/core/Raycaster.intersectObject

Currently, the A-Frame raycaster component ignores this value.

This means that when a ray moved from hitting one member of an instanced mesh, to hitting another, there would be no raycaster event emitted, and hence no way for an application using A-Frame raycaster to update its state.

Proposed change

Proposed change is that A-Frame raycaster would pay attention to the instanceId value on an intersectObject returned by the Three.js raycaster, and whenever the instanceId value changes, issue raycaster-intersected/intersection-cleared & event, followed by a raycaster-intersected/intersection event with the new raycast information.

It would be up to the application to interpret this data, making use of the instanceId value.

There should be no impact on behaviour for any entity/object that does not use an Instanced Mesh.

** Next steps **

I'd be happy to do an A-Frame PR for this. The changes aren't trivial, but they don't look to be too complicated either. the raycaster component already keeps track of the previous raycast intersectObjects, so it has access to the state it needs to implement the change.

Along with this PR, I'll make enhancements to my instanced-mesh component to receive the additional events & handle them appropriately, so that I can have a working proof-of-concept, and confidence that the code works.

But before I go ahead with this, I'd appreciate an indication that you'd be happy in principle to take this change into A-Frame, once I make it all work).

[dmarcos]

Yeah. Thanks for suggesting. Hopefully we can keep the raycaster changes to a minimum. While a cool feature, instancing is pretty niche at the moment and concern is to make raycaster code much more complicated and harder to maintain.

[dmarcos]

If it’s not clear from my comment. I’m open for the raycaster to consider instancing if we can keep it simple 🙂 Re-read my comment and thought it might not be clear

[arpu]

@diarmidmackenzie do you have a DRAFT for this? this would be very useful

[diarmidmackenzie]

I don't think I ever wrote the code changes. I eventually decided this was unecessary and over-complicated. My raycasting examples in instanced-mesh don't use this.

E.g. https://diarmidmackenzie.github.io/instanced-mesh/tests/raycasting.html

Here's some notes on why... https://diarmidmackenzie.github.io/instanced-mesh/#:~:text=does%20support%20raycasting%20directly%20against%20the%20instanced%20mesh.%20That%20functionality%20is%20not%20used%20by%20this%20implementation%20for%20several%20reasons%3A

[polytropoi]

@arpu here's an example of instanced mesh raycasting, that just gets the instanceID. https://github.com/polytropoi/ServiceMedia/blob/0c948fccc6d38c14a52029333c3df458eb727ea5/main/src/component/mesh-utils.js#L1119 e.g. https://smxr.net/webxr/0Og9lT93J Note that you do need to use the three.js raycaster, though. See @diarmidmackenzie's repo for a complete solution, that works with physics.