NASA-AMMOS
Allow higher zoom level overlay than "master" tileset tile's using ImageOverlayPlugin
Description
When overlaying aerial imagery on quantized mesh, for instance, I find the resolution of the texture to be lower than personal preference. I can reach the max zoom level of the overlay, but only when I am extremely zoomed in (almost not being able to see anything, being so close). errorTarget=1, if it helps; also tried with errorTarget=0 with same results.
My understanding is that the plugin uses the same zoom level for the overlay as the "master" tileset's current tile, and only uses a higher zoom level for the overlay in case the "master" tileset's max zoom level is reached and there is still a higher zoom level available for the overlay (and enableTileSplitting is set to true).
I personally find it to be too low, at least with Swiss dataset: Quantized mesh Aerial imagery
See fiddle here.
Solution
I would like to have a way to force using a higher zoom level for the overlay than the underlying tileset's tile.
I know 3D tiles work with setting errorTarget, but here I am not sure if that would make sense to have an errorTarget for the overlay as well.
Maybe having a zoom level offset? If zoomLevelOffset=2, for instance, the overlay zoom level will be picked at "master" tileset tile's zoom level + 2 (if the 3D tile or quantized mesh is zoom level 18, it will overlay the tile with zoom level 20 imagery).
Alternatives
None. Living with it :)
Additional context
I have much better overlay resolution using TextureOverlayPlugin. Live demo available here. Look for "Vevey" in the search bar top right to see the same location as the fiddle. Screen capture:
You've mentioned it in the other issue but this should be fixed with https://github.com/NASA-AMMOS/3DTilesRendererJS/commit/8266a35af1db54b22589158f07f7510c030eaac9 & #1274. That bug in effect meant that "enableTileSplitting" wasn't working when used with quantized mesh so higher levels of detail would not be visible because the tiles fail to split. It would be good to know if this is still an issue in the master branch.
I know 3D tiles work with setting errorTarget, but here I am not sure if that would make sense to have an errorTarget for the overlay as well.
Can you clarify whether the issue is that more data doesn't seem to load as you zoom in? Or whether not enough detail seems to be displayed even when the tiles aren't disappearing?
errorTarget=1, if it helps; also tried with errorTarget=0 with same results.
Just a note that errorTarget=0 probably never really be used. This will cause a load of a ton of data, possibly filling up the data cache and result in the appearance of loading taking a long time, likely giving a false impression of the level of data that can be displayed.
See fiddle here.
Somewhat of an aside but I'm seeing some "twisting" of the camera in this fiddle when zooming in / out. This may be a bug relating to Z or non-X rotations of the target ellipsoid 🤔 I'll have to take a look another time.
No, I think you misunderstood this "issue". This is actually a feature request, thus the "enhancement" label. I am not reporting a bug.
What I would like is to be able to say that even though I don't necessarily want the 3d buildings to be loaded at high LoD, I want the aerial texture to be loaded at the, say, max LoD.
Bug(s) aside, currently, if the 3d buildings tile is loaded at zoom level 10, the aerial imagery is loaded at zoom level 10 as well. I would like to have the overlayed aerial imagery to be loaded at zoom level 20 and keeping the 3d buildings at zoom level 10.
I hope this makes more sense.
Can you clarify whether the issue is that more data doesn't seem to load as you zoom in? Or whether not enough detail seems to be displayed even when the tiles aren't disappearing?
As of this current "issue", I am not reporting any problem with zooming or tiles load.
Just a note that errorTarget=0 probably never really be used.
Understood. I am just trying to have as much detail available as further as possible on the horizon. Thought this would help.
Somewhat of an aside but I'm seeing some "twisting" of the camera in this fiddle when zooming in / out.
Yes, I noticed it as well. This is not my main priority now, as I don't have this issue in my app. This is just a quick fiddle to illustrate.
I would like to have the overlayed aerial imagery to be loaded at zoom level 20 and keeping the 3d buildings at zoom level 10.
Thanks I understand, now - as you're seeing the mapping of tile overlay level to tile is fairly naive, just using the tile depth to select the image level to load. This can lead to a few different issues including a large visual discrepancy when sibling tiles have drastically different geometric error resulting in them loading at different rates.
You can see this in the Swiss tiles data set where one tiles geometric error at level 10 is set to ~4 while others are set to ~150 resulting in that "4" tile not refining and displaying a lower resolution texture than other tile around it that have refined. It should be noted that this swiss tile set is a bit odd and is only defining "geometricError" for a subset of tiles so the discrepancy is likely in part due to the auto-generated GE being different from metadata definitions. Perhaps there's a better way to handle that but the spec is non specific here and either way it illustrates the point:
On to what can be done about it - the first step would probably be to improve the default behavior and more effectively select an appropriate image tile level that sensibly maps to the tiles level of geometric error. It's not neessary straight forward to explain but one approach for this may be to pick the physical, projected size of a pixel for the texture used to cover the span of the tile based on the tiles geometric error and then load the tile that will fill that texture in given the resolution image tile set. Then from here if more detail is still needed we could discuss factors or other user-facing scaling for increasing this.
The idea is the treat the world-space size of the pixel as "geometric error" assuming that an infinitely small pixel would be 0 error. The image tiles plugin (as used in this demo) use the same idea for calculating tile geometric error for the images and works reasonably well, I think.
For reference the function used to select the image layer to use is here.
It should be noted that this swiss tile set is a bit odd
You mean the ones for the 3d buildings here, right? Yeah, based on your inputs, I have quite a few "complaints" for them, but they don't seem to be very keen to update their workflow here...
The idea is [...]
Do you expect me to submit a PR?
Yes, for ImageOverlayPlugin I found where the level is selected for the purpose of another issue I submitted, but I see the demo your shared uses XYZTilesPlugin and this is a bit more difficult for me to identify where it is defined for this plugin, with the ImageFormatPlugin inheritance, the XYZImageSource, etc.
I also found in TiledImageSource a TODO you already have which seems to match your proposed approach:
// TODO: support queries for detail at level - ie projected pixel size for geometric error mapping
Thank you for the insights when you have time.
Do you expect me to submit a PR?
I know it's not always obvious what the source of the current behavior might be so I generally just wanted to elucidate what's going on so you or others could add thoughts or understand what's happening and why (or for myself in the future). PRs are always appreciated if this is something you want added - I otherwise probably won't be able to prioritize in my free time anytime soon.
but I see the demo your shared uses XYZTilesPlugin and this is a bit more difficult for me to identify where it is defined for this plugin
For planar data sets the geometric error is calculated here. The full tile set is loaded to fit in a 1x1 unit (with some exceptions) so the "world scale" of a pixel is 1 / pixelCount.
For the the ellipsoid image plugins the calculation happens here and is a bit more complicated. Basically the world space "width of a pixel" is calculated based on the latitude and largest circumference of the ellipsoid where the tile is at since this is where pixels will be most stretched.
// TODO: support queries for detail at level - ie projected pixel size for geometric error mapping
Thank you for the insights when you have time.
Yeah I think that comment can be moved or removed - the idea was to allow for querying a level at a specific pixel density to support the concept outlined in https://github.com/NASA-AMMOS/3DTilesRendererJS/issues/1278#issuecomment-3200824139. Ultimately I think this belongs in TilingScheme, now.
Thinking this through a bit more - this can be a bit difficult to design this so it "just works". It's possible to infer something akin to a "geometric error" but that doesn't mean it notionally maps to the levels of detail of the underlying geometry since "geometric error" for tile sets is often relative to the highest level of detail which means apparent detail can change dramatically between tile sets. Directly mapping the geometric error values effectively "pins" the LoDs a geometricError === 0 which could result in a ton of tiles being loaded for higher geometry.
Pinning the geometric error of the "root" tile, though, is unreliable since tile sets can have extremely high geometric error at the root that makes big jumps for subsequent child tiles.
There could be a mix of clamping the number of tiles to load / render for a specific geometry tile, detecting existing texture sizes, or pinning world-space texel sizes along with geometric error mapping may be some options to consider?
Hi @gkjohnson, Happy to see you have again a bit of time to dedicate to this awesome library. I am also back working on this project and hope we can synchronize to work on a few things together :) Thank you for giving another look at this issue. Honestly, I think my level of understanding of the library is not sufficient to get why geometric error is applicable to the overlay tileset, since for me it is a 3D tiles concept. From what I can see, the selected zoom level of the overlaid tile is the same as the underlying 3D tiles'. Maybe this approach is too naive, but in my opinion we should focus on texture size as you proposed. In a WMTS tileset for instance, we know what is the pixels and meters sizes of the tiles of a zoom level. Maybe we can find some sort of mapping between the 3D tile's geometric error and the expected overlaid image resolution?
Sorry in advance for the long response - just trying to get some thoughts out and this is not necessarily an intuitive topic 😅
Thank you for giving another look at this issue. Honestly, I think my level of understanding of the library is not sufficient to get why geometric error is applicable to the overlay tileset, since for me it is a 3D tiles concept
There are a few reasons - fundamentally what we're trying to is align both the overlay detail and the 3d tiles geometry so that they increase at a rate that feels synchronized to the user. 3D Tiles increase in detail visually based on the geometric error, which is not guaranteed to grow at the same rate as their depth in the tree (eg there can be significant or minimal jumps in geometric error between tile levels or even siblings).
On the other end, image tile detail typically doubles per level so it's very consistent. Likewise the "highest" level of detail ("zero error") can be at significantly different depths between the formats. So in this case simply mapping image levels to some form of tile depth while geometry tiles are hiding and showing using geometric error means you can wind up in cases where sibling geometry tiles have significantly different GE resulting in significantly different hierarchical levels being displayed next to each other so you'll get the types of detail discrepancy shown in https://github.com/NASA-AMMOS/3DTilesRendererJS/issues/1278#issuecomment-3200824139. So really we want to find a way to make sure that we're displaying some kind of consistent level of detail for image overlays on the screen, and geometric error is the driving factor for determining what to display.
One of the other issues is that even if we can come up with some model for mapping the image tiles the "detail granularity" of image tiles may not always be consistent due to things like text sizes embedded in the images. And geometricError is not always used with a consistent definition in tile sets or may not lend itself to this kind of mapping so it cannot necessarily be relied on to map well. All the geometric tiles in your tile set from #1280, for example, have geometric error 0 which would be difficult to implicitly map.
I'll have to think through using texture sizes and geometric error a bit more and write my thoughts later (though of course I'm open to other perspectives here, too) but I think it's clear regardless that there is no "perfect" general solution so there will need to be some kind of user configurability here.
I'd be open to adding a user-settable callback so there's freedom to set the depth of tile to use directly so users can adjust their overlays for their specific use cases as-needed. We can consider this "experimental" as we discuss some of the other implications around texture sizes, etc, since it may cause the callback to have to change. Something like so:
tiles.registerPlugin( new ImageOverlayPlugin( {
renderer,
overlays: [ new XYZTilesOverlay( {
url: 'https://tile.openstreetmap.org/{z}/{x}/{y}.png',
mapZoomLevelToTile: tile => tile.__depth + 1,
} ) ],
} ) );