Undefined terrain picks in 2D/CV mode.

[mzschwartz5]

What happened?

In 2D / CV modes, many picks are undefined when they shouldn't be. (See comments below for more info - original issue description misattributed issue as a regression from recent PR).

~This comes from the terrain picking performance PR, which now uses an incrementally constructed quadtree to accelerate picks.~

~1. It does not happen when the quadtree is limited to a single (root) node. 2. The fact that it does not happen in 3D mode tells me the issue is likely related to the computation or use of the 2D transform here.~

edit- that... no longer seems to be true. Not sure if I'm doing something different now or then, but even if I set the maximum quadtree level to 0, the issue still occurs now.

Reproduction steps

1. Sandcastle demo
2. Open console
3. Zoom out
4. Move mouse, observe undefined logged in some regions of the map....

Sandcastle example

No response

Environment

Browser: CesiumJS Version: Operating System:

[mzschwartz5]

Forum post where the issue originated

[mzschwartz5]

Somehow, the issue is actually occurring in this function - which is not finding any surface tiles that intersect the given pick ray.

[mzschwartz5]

Okay actually... I've been barking up the wrong tree entirely. I reverted #9961 locally and tested again, and the issue still occurs.

I'm not sure why I thought otherwise, earlier. (I hope I'm not being fooled by some sort of caching here). But I think this is actually a long-standing issue in 2D / CV modes, not a regression. And the issue appears to be in the function linked above - in finding which tiles intersect the pick ray. Either the pick ray must be ill-formed, or the bounding spheres of the tiles are ill-formed.

[mzschwartz5]

A little closer to the root cause: in my last comment, I said it's either the pick rays or the tile bounding spheres causing the issue. In fact, the issue is actually higher upstream. First of all, it seems the undefined issue seems to occur much more often when zoomed all the way out in 2D mode.

In Globe.pickWorldCoordinates, we iterate over each tilesToRender (code). I logged these tiles as I zoomed out, and found that, right when the undefined pick results start to occur, tilesToRender switches to a different set of tiles (and ones that don't make sense for what is on screen - cross checked using the cesium debug inspector).

If I move the screen over some, I can get defined pick results at the same coordinates, because the tilesToRender have changed.

So the issue isn't the bounding sphere calculations or the ray formation, it's that the reported set of tiles to render is incorrect... which is weird because we're definitely rendering the right tiles. Will have to investigate further.

(In the image below, I'm logging (X, Y, Level) of the tilesToRender - note how far off it is from what's displayed on screen)

[mzschwartz5]

So I've been searching through QuadtreePrimitive.js looking for answers.

With some debugging, same setup as above, I see that the quadtree traversal visits some large # of tiles, which contain the tile on screen above, but only 4 of them make it into tilesToRender - the left column (x = 7). Further, if I turn on suspendLodUpdates in the inspector, everything except those 4 tiles disappears:

Not exactly sure what this tells me yet... but the suspendLodUpdates flag prevents selectTilesToRender from running each frame and uses the cached tilesToRender.... so when it's true, I see those 4 tiles on screen, but when it's false, something else is dictating what to render...

[kring]

I see those 4 tiles on screen, but when it's false, something else is dictating what to render...

I think this is related to antimeridian wrapping. I'm not sure offhand how it works, but clearly something is rendering a single tile at two different locations. Notice there are two Africas:

Probably turning on suspend update disables whatever is doing that, and so we only see the tile at the last location.

It's also possible (likely) that the picking isn't accounting for this process.

[kring]

Here's the code that renders twice in 2D, with two different viewports: https://github.com/CesiumGS/cesium/blob/1.136/packages/engine/Source/Scene/Scene.js#L3283 Looks like it runs the tile selection algorithm twice, too. So that's why suspend update loses tiles; it only has the tiles selected in the second run. And probably why picking is unreliable, because it only looks at the current tilesToRender rather than running the selection algorithm and picking twice.

[mzschwartz5]

Good insights @kring, thank you. I think you're exactly right. Maybe we can just store all tiles rendered in a single pass on the QuadtreePrimitive.

Separately, looks like I still have a little work to do- Kevin's observation only applies to 2D mode, but I know this issue happens in CV mode as well. Playing around a little more, it seems the problem happens more intermittently when zoomed in - both in 2D and CV modes, and I think it's a separate cause. That may be a regression from the fast terrain picking PR after all.

[mzschwartz5]

I prototyped a fix (basically what I said above - just collect all tiles from each QuadtreePrimitive render call each frame, and iterate over that in Globe.pick).

It does get the correct tiles now, but the issue persists. I think it's still related to antimeridian wrapping. I set a conditional breakpoint for a given tile in Globe.pick; compare its bounding volume to the pick ray (when hovered above said tile):

Bounding Volume:

center = (0, 12523442.7..., -2504688.5...)
radius = 3542164.5...

Pick Ray:

direction = (-1, 0, 0)
origin = (12756274, -27520248.5..., -2433244.7...)

The ray's height (X coordinate in 2D mode) is above the bounding volume, looking down ✅. The Z coordinate (which is "2D" for what we usually think of as "Y") is well within range ✅ . The Y coordinate ("2D" for "X") is wayyyyy off.

So now the question is- what should change? The 2D bounding volume calculation, or the pick ray calculation?

[mzschwartz5]

Okay it does seem like a bug in the ray calculations regarding the antimeridian. Check this out: I'm grabbing a point on the map and dragging. Because my mouse is staying at the same position relative to the map, I would expect the pick ray to stay the same. It does - until crossing a certain threshold, at which point it seems to underflow.

(Logging ray.origin.y / 1million) https://github.com/user-attachments/assets/cc16ea0c-e565-4769-a7d8-75ff537c7773

[mzschwartz5]

This issue is only present in 2D scene mode + MapMode.INFINITE_SCROLL. If MapMode is ROTATE, the map does not wrap around, and the issue doesn't exist. The issue is that, when the map wraps around, the camera's world position wraps with it, but the mouse position does not account for that.

See code here. To put some numbers to it, the camera's horizontal world position ranges from (roughly) 20 million meters to -20 million meters before wrapping around. But if the camera is at 20mm, and the mouse is off to the right of that, this function may return something like 30mm for the pick ray origin - an invalid world coordinate.

(Note that this doesn't entirely account for the undefined issues.... this is just the 2nd of three (or more..) issues identified so far)

[mzschwartz5]

I fixed the two issues (ray generation and rendered tiles) locally. Then I tested with and without the fast picking terrain PR (i.e. I reverted that PR locally). The undefined issue happens with and without it, still - but it's more prevalent with it. Without it, the issue only happens at the antimeridian. With it, it still happens there, but also intermittently (probably near boundaries of the terrain picker's quadtree AABBs).

So - that means we still have two more issues (at least) to track down here.

(Note that the antimeridian issue occurs even in MapMode.ROTATE (not just in INFINITE_SCROLL). It happens near the meridian (left side of the map in ROTATE mode). It also happens near the poles!)

[mzschwartz5]

Fixed the antimeridian issue with a little modulo math (it seems in 2D mode, the vertex positions near the antimeridian could wrap to the other side of the globe even though visually they're still on this side. By comparing to the ray origin, we can adjust those positions to be in the same wrap interval).

Still having trouble with undefined pick results around the poles. I suspect it may have to do with the fact that triangles at the poles are dense and skinny. When projected, they may become degenerate. I'm not how to fix it, or if it's even worth fixing, to be honest. It's less noticeable than the antimeridian issue was.

Also still need to investigate the issue introduced by the quadtree picker.

[mzschwartz5]

A sandcastle I'm using to try to understand if there's a pattern to the undefined pick results attributable to the fast terrain picking PR.

It looks like my original PR description (now crossed out) once again stands: the picking issues seem to stem from child nodes. If I set the max depth of the quadtree to 0 (a special case which is identical to the non-quadtree approach), then no more undefined points.

Worst case scenario if I can't root cause this in a timely manner - we can expose the quadtree depth setting to users so they can set it to 0 to disable the optimization for 2D mode. We should probably expose the setting anyway.

[mzschwartz5]

Okay I think I've finally figured out another particular issue with picking in 2D. Or, at least, I've narrowed it down a lot. In 2D/CV modes, some tiles seem to be passing this check to compute a 3D tile transform instead of the 2D one.

edit- I think this may be a result of this comment edit2 - yep, that's it. Some small adjusments to the guard in TerrainMesh.getTransform solve the issue.

[mzschwartz5]

All fixes are in place in #13098 - 2D picking is now much more robust (the only remaining issue area is at the poles).