`PickingInteraction` randomly switch to `None` even when hovered

[djeedai]

Bevy version

0.16

What you did

/// Workaround for #19464 change detection broken on [`PickingInteraction`].
#[derive(Component)]
struct PrevPickingInteraction(pub PickingInteraction);

fn save_prev_picking_interaction(
    mut query: Query<(Entity, &PickingInteraction, &mut PrevPickingInteraction)>,
) {
    for (e, pi, mut ppi) in &mut query {
        trace!("hover: e={:?} prev={:?} cur={:?}", e, ppi.0, *pi);
        ppi.0 = *pi;
    }
}

app.add_systems(First, save_prev_picking_interaction);

What went wrong

The target Mesh2d is a large-ish rectangle, so there's no change the mouse accidentally leaves it. When slightly moving the mouse around a few pixels, I randomly get a None for 1 frame, immediately followed by a Hovered again. This looks like a race condition or timing issue where 1 frame would be skipped for example, and would not detect the mouse temporarily. I'm already working around #19464, and now it seems I also need averaging/smoothing over multiple frames to get a stable hover/non-hover (mouse enter/leave) pair of events. This makes using bevy_picking pretty cumbersome.

[djeedai]

Here's an example terminal output. It's easily visible because of the regularity of lines when everything works well, and the occasional visual difference when a frame is skipped:

("hover" runs in First before update_interactions(), while "highlight" runs after in Update)

2025-06-02T09:34:00.662835Z TRACE demomino: hover: e=9v1#4294967305 prev=Hovered cur=Hovered
2025-06-02T09:34:00.663942Z TRACE demomino: highlight: e=9v1#4294967305 no-op pi=Hovered
2025-06-02T09:34:00.679005Z TRACE demomino: hover: e=9v1#4294967305 prev=Hovered cur=Hovered
2025-06-02T09:34:00.680134Z TRACE demomino: highlight: e=9v1#4294967305 no-op pi=Hovered
2025-06-02T09:34:00.695653Z TRACE demomino: hover: e=9v1#4294967305 prev=Hovered cur=Hovered
// ****** Here the update_interactions() system changes Hovered->None for no reason
2025-06-02T09:34:00.696798Z TRACE demomino: highlight: e=9v1#4294967305 pi=None
// ****** Next frame my system picks up the change
2025-06-02T09:34:00.711972Z TRACE demomino: hover: e=9v1#4294967305 prev=Hovered cur=None
2025-06-02T09:34:00.713156Z TRACE demomino: highlight: e=9v1#4294967305 pi=Hovered
2025-06-02T09:34:00.728982Z TRACE demomino: hover: e=9v1#4294967305 prev=None cur=Hovered
2025-06-02T09:34:00.730242Z TRACE demomino: highlight: e=9v1#4294967305 no-op pi=Hovered
2025-06-02T09:34:00.744926Z TRACE demomino: hover: e=9v1#4294967305 prev=Hovered cur=Hovered
2025-06-02T09:34:00.746081Z TRACE demomino: highlight: e=9v1#4294967305 no-op pi=Hovered

I don't have a strong proof that update_interactions() is responsible for the change but I don't see what other system would write that PickingInteraction value (none of mine).

[djeedai]

I tried to debug a bit more, although it's very hard to see what's going on as it depends on the position of the mouse, which is not constant when you pause with a debugger.

It seems that indeed update_interactions() changes to None despite the mouse hovering. It doesn't look like there's anything else interacting with PickingInteraction. I assume this is a bug in the mesh picking backend where for some reason it fails to "see" the mesh for a frame, maybe a race condition in system ordering or a logic bug. But again, it's very hard to tell for sure unless we had tracing maybe. Trying to debug with a local copy of Bevy (with [patch.crates-io]) doesn't work; both the local and official crates build, and then conflict with each other giving tons of build errors.

[djeedai]

Ok so while trying to make a repro, I found that my background rect behind the pickable buttons/tiles is also made pickable by bevy_picking. So the working theory, and workaround, is:

• 2D ortho camera
• background and buttons at same depth (Z=0)
• rendering of background uses custom material, so can't be batched with buttons, which indirectly forces some kind of sorting
  • confirmed that removing the custom material and all textures, rendering is also z-fighting
• moving the background e.g. Z=-1 fixes the picking

So all signs point at unstable picking order + default everything pickable, the latter being IMHO a very bad default behavior (why would any game want by default 100% of its meshes be pickable, it's highly unlikely).

[aevyrie]

why would any game want by default 100% of its meshes be pickable, it's highly unlikely

It probably wouldn't. The game would probably be using a physics engine to drive picking, and that backend would probably be opt-in per entity. I believe the rapier and avian picking integrations behave this way. The mesh picking backend was added in large part for debugging/testing picking in 3d, which is one reason why it isn't added by default.

[djeedai]

You can't add a built-in API and ship it with Bevy, and tell the user they shoot themselves in the foot by trying to use it because they should have instead used a third-party crate.