Heightmap terrain data visually sinks downwards when upsampled past its limits

[abhimonk]

When heightmap terrain is upsampled past its available data, it appears to "sink" into the ground and decrease in height with each upsampling. Here is a gif of the issue:

(I am unable to provide a sandcastle link as I am not able to share the terrain data, apologies).

I have a heightmap terrain dataset on a server and we use CesiumTerrainProvider with a url to our heightmap server. The dataset only goes up to zoom level 10, but I have basemap imagery that goes much higher (i.e satellite imagery that goes to zoom level 20).

If I restrict the terrain tiles to only go up to level 10 (shown in the above gif via the Cesium inspector), then a milk truck placed exactly at the terrain height appears to be perfectly on top of the terrain (note the back wheel is just barely clipping into the terrain, which proves it is perfectly on top of the terrain).

If I then enable LOD update (which allows Cesium to upsample our heightmap terrain past its limits to higher zoom levels), I see the terrain sink downwards below where the actual data is, and it worsens with each additional upsampling. This makes the truck look like it's floating.

I'm aware that other artifacts, like the cracks/rifts and undulations are a result of upsampling heightmap data, but the sinking of the terrain is very difficult to deal with, as it makes all ground vehicle heights look incorrect (even if they are placed correctly according to the underlying data).

The issue does not occur when I try it with a different dataset that uses quantized mesh instead of heightmap, but I am currently unable to switch to quantized mesh for our main dataset, and would like to see if there is a workaround for this issue using heightmap data.

Things I have tried:

• Manually shifting the terrain upwards by just adding a fixed offset value. I do this by increasing the terrain provider's heightmapStructure offset, or by overriding the terrainprovider.upsample function and adding a flat value to the terrain data being shown. This workaround doesn't work for us as the terrain 'sink' amount doesn't seem to be the same everywhere, so one flat offset wouldn't handle all cases.
• Restricting upsampling such that it does not go past our terrain zoom level. This workaround gives us the desired outcome with respect to the terrain data, but has the side effect of restricting our basemap's zoom level too, making high resolution basemap imagery clamped to only level 10, which is not desireable for us.
• Writing a custom terrain provider class that has its own requestTileGeometry function that tries to change the way the data is given after zoom level 10. This required that I configure the provider to believe it has more than level 10 data (otherwise requestTileGeometry doesn't get called past our max level of 10), but after this, I realized I would have to manually upsample our own data which looks like what the built-in upsample function already does.

Is there any workaround or solution for this issue that doesn't involve switching our large dataset from heightmap to quantized mesh? (i.e Is there a way to restrict Cesium from upsampling our terrain data while still allowing it to display higher resolution basemap imagery on top of it? Or a way to prevent it from sinking when upsampling?)

Reproduction steps

1. I have a heightmap data set on a server. The data only goes to zoom level 10
2. I add this data using CesiumTerrainProvider with a URL.
3. I use a basemap imagery data set that goes up to zoom level 20
4. When I zoom in, cesium upsamples my terrain data (presumably to match the imagery) and the terrain sinks lower and lower into the ground, below where the actual data is.

Sandcastle example

I am unable to provide a sandcastle link as I cannot share our heightmap terrain data.

Environment

Browser: Chrome CesiumJS Version: Operating System: Windows

[lukemckinstry]

Thanks for your detailed post describing the problem you are facing and steps you have tried. It would be helpful to have sample terrain data the reproduces the error so we could debug if there is a subset you were able to provide (third party data would be fine too). Are you seeing this as a regression from previous versions of Cesium?

We have a concept of maximumLevel and minimumLevel for our imagery providers https://cesium.com/learn/cesiumjs/ref-doc/ImageryProvider.html?classFilter=imageryprovider but I am not sure why we do not provide this for terrain providers.

Since you mentioned quantized mesh, I'll mention we do encourage quantized mesh over heightmap terrain, but as the docs indicate, CesiumJS still supports heightmap terrain. So please let me know about the questions above regarding reproducing sample data and if this is a regression, and I can try to track down better answers for you.

[abhimonk]

Thanks for the response. I don't think this is a regression from a previous version. We've seen this issue from back when we first started using Cesium around version ~1.60 or so. It's been causing us more problems recently though as our recent workflows involve placing 3D models on the surface of the terrain.

I have received permission to share the 0/0/0 and 0/1/0.terrain files, so I can provide a sandcastle repro below (though it requires running a local server, which is included in the zip file below). See the below reproduction steps to download level 0 of our heightmap terrain data along with the layer.json and server.py.

The reproduction steps:

1. Download this zip file, containing our level 0 heightmap terrain data, the relevant layer.json file, and a server.py CesiumHeightmapRepro.zip
2. Run server.py from that directory via "python server.py"
3. Run the following sandcastle: Link
4. By default, it uses Cesium's terrain. Switch to "local terrain" using the top-left UI dropdown. You may need to jostle the camera a bit afterwards to force it to update).
5. Use the cesium inspector terrain > show tile coordinates.
6. Zoom out until you hit something under zoom level 10, or even higher.
7. Use the inspector to suspend LOD updates (Terrain > suspend LOD updates)
8. Zoom back in to that baseball field in san fransisco south of the bridge
9. Right-click to put down a milk truck. Looks good! seems to be roughly on the terrain
10. Un-check "suspend LOD updates" and watch the terrain sink as it gets upsampled. Now the milk truck appears to float.

[lukemckinstry]

Thanks for the data and reproduction steps, I can confirm the behavior you described for the custom heightmap terrain case. And for comparison, it appears that the CesiumWorldTerrain works as expected for the test steps. I am working on learning if this is a known limitation or bug, and if there is a workaround available. Setting model.heightReference = Cesium.HeightReference.CLAMP_TO_TERRAIN does not seem to help.

[kring]

I don't have a completely satisfying explanation yet, but I think it probably has to do with the coordinate system in which the interpolation for upsampling is performed. We interpolate heights for heightmap upsampling in longitude/latitude space. Which is not the same space in which the triangles are drawn. Imagine we have a tile that looks like this, with four heights which are connected together to form two triangles:

To upsample, we want to divide this tile into four smaller tiles at the red dot. The longitude of that red dot is simply (westEdge + eastEdge) / 2, and the latitude is (southEdge + northEdge) / 2. Now what's the height? I believe the heightmap upsampling computes it as (southwestHeight + northeastHeight) / 2. Seems reasonable enough.

Except this square is not really a square when it's mapped to the globe. When we render these triangles, they're connected by a straight line in Cartesian ECEF coordinates. That line will not even pass through our computed longitude/latitude center point! And if we were to carefully compute the height at the longitude/latitude we've chosen as our center point, its height would not be the same as we computed above.

This problem is very obvious in extreme cases like in the test case described here. Where we only have terrain data for level one, where height samples are over a hundred thousand meters apart, and then we're looking at it really up close where we can observe a one meter height difference.

In the more common case where we don't start upsampling until we're zoomed in pretty close, it's usually much less of a problem. As we zoom in closer, interpolating in longitude/latitude space becomes "less wrong" because the relationship between ECEF and longitude/latitude/height gets much closer to linear.

That's why the explanation here isn't totally satisfying. I wouldn't expect major problems starting interpolation with level 10. So there could be something else going on here, I'm not sure yet. It would definitely help immensely to have a more realistic test case to work with. Even if it's created with just some random public data from the internet.

[abhimonk]

Thanks again for the response. I have a new repro using 10 levels of terrain data:

1. Download this zip file: CesiumHeightmapMultilevelRepro.zip, and run server.py in the same way as the previous version. I created this heightmap data using DTED level 2 data from the USGS earth explorer, and trimmed it down to just the one tile at each level that we care about, and deleted all levels higher than 10. I believe this is the most similar case to our internal data that I can provide using public data.
2. Run this new sandcastle and repeat the steps from the previous instructions. The only difference in this sandcastle is that placing the milk truck samples from the terrain at level 10 (the highest level offered by the local data) when using local terrain instead of 0 like the previous example.

Just like the first example, when I repeat the steps, the milk truck appears perfectly placed when I limit LOD updates to level 10, but as soon as I uncheck that box and allow LOD updates past 10, the terrain sinks downwards when using the local heightmap terrain data.

Let me know if you can reproduce the issue with this, and if you have any other insights or ideas for workarounds. Thanks!