laxnpander
DRAFT: Tiling Stage Fixes and Improvements
Description
Fixes to numerous small bugs and odd behaviors discovered when trying to use the tiling stage for the first time in a while.
There are also some improvements to make the output from the tiler more compatible with different tile viewing tools, including those that doesn't implement the TMS standard.
Reason
- There are a few locations where the code crashes when creating higher zoom tiles
- Higher zoom tiles have their valid data overwritten with blank data both in rgb and elevation_angle files causing zooms above max to be missing a lot of data
- There also appears to be a small shift in the x axis relative to the full mosaic at different zoom levels. This may just be pixel rounding, or may be something else. It is TBD.
Method / Design
- Perform a check in cv_grid_map that the merged tiles will fit inside the pixel space of the roi's
- Attempt to handle NaNs in floating point matrices better to avoid ambiguities
- Update blend routines to match those used in mosaicing. The older method had translucent pixels around the stitch location.
- Add a
use_tmsflag to the tiling.yaml file to allow using TMS, or Google/OSM standard tile names in the cache. This primiarily inverts the y axis. See here: https://www.maptiler.com/google-maps-coordinates-tile-bounds-projection/
Testing
Compiled and run on:
- Ubuntu 20.04 - Desktop
- Custom OpenREALM wrapper (non-ROS)
- Fed results of ortho stage to both mosaic and tiling for comparisons
Other Notes
@zthorson One thing about the shift: Did you realize that the tileing is using EPSG:3857? This is slightly different from UTM coordinates and could be the root of the shift depending on the visualization that you use.
Edit: As a frame of reference I use this file for my dataset to visualize the tiles. It must be copied into the folder where the different zoom levels are.
<!DOCTYPE html>
<html>
<head>
<title>ortho</title>
<meta http-equiv="imagetoolbar" content="no">
<meta http-equiv="Cache-control" content="no-cache">
<!--meta http-equiv="refresh" content="0.2"-->
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.3.1/leaflet.css">
<link rel="stylesheet" href="https://cdn.rawgit.com/ardhi/Leaflet.MousePosition/master/src/L.Control.MousePosition.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.3.1/leaflet.js" type="text/javascript"></script>
<script src="https://cdn.rawgit.com/ardhi/Leaflet.MousePosition/master/src/L.Control.MousePosition.js" type="text/javascript"></script>
<style>
body { margin:0; padding:0; width:100%; height:100%; background: #ffffff; }
#map { position:absolute; top:0; bottom:0; width:100%; z-index: 1; }
#slider{ position: absolute; top: 10px; right: 10px; z-index: 5; }
</style>
</head>
<body>
<div id="map"></div>
<!--input id="slider" type="range" min="0" max="1" step="0.1" value="1" oninput="layer_rgb.setOpacity(this.value)"-->
<script type="text/javascript">
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
var generateRandInt = function() {
return Math.floor( Math.random() * 200000 ) + 1;
};
var mapExtent = [10.22556861, 52.40028879, 10.23224548, 52.40437500];
var mapMinZoom = 15;
var mapMaxZoom = 20;
var bounds = new L.LatLngBounds(
new L.LatLng(mapExtent[1], mapExtent[0]),
new L.LatLng(mapExtent[3], mapExtent[2]));
var map = L.map('map').fitBounds(bounds);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png').addTo(map);
var options = {
randint: generateRandInt,
minZoom: mapMinZoom,
maxZoom: mapMaxZoom,
opacity: 1.0,
attribution: 'Rendered with OpenREALM>',
tms: true
};
var layers = [L.tileLayer('{z}/{x}/{y}.png?{randint}', options).addTo(map)];
setInterval(function() {
layers.push(L.tileLayer('{z}/{x}/{y}.png?{randint}', options).addTo(map));
if (layers.length >= 5) {
layer_deleted = layers.shift();
map.removeLayer(layer_deleted);
}
console.log(layers.length)
}, 200);
</script>
</body>
</html>
@laxnpander I was using QGIS to visualize the tiles, but it seems like that file may work better, as I won't have to deal with the caching that QGIS uses.
I'm noticing a nice memory footprint improvement using the tiling stage instead of the mosaicing one that I'm hoping to test on a device soon as well.
@zthorson In theory, memory required should be more or less constant as all unnecessary data is being written to the disk and removed from the RAM and only relevant tiles dynamically loaded. However, this of course comes with more read/write operations and possibly a significantly higher CPU load. So you will have to check whether this works for you or not.
In case you hit a performance sink somewhere, I have two things in mind where you might want to make adjustments:
- The tile cache has a rather simple caching strategy. I basically check what ROI was requested last time and I try to predict what tiles might be new per zoom level: https://github.com/laxnpander/OpenREALM/blob/465ecc8a68a55dd7fee956af01b56169ea6db231/modules/realm_ortho/src/tile_cache.cpp#L372 I guess if you put more thought into it, you could come up with a more CPU friendly strategy.
- GDAL warper is ridiculously slow https://github.com/laxnpander/OpenREALM/blob/465ecc8a68a55dd7fee956af01b56169ea6db231/modules/realm_ortho/src/gdal_warper.cpp#L169 The transformation into EPSG:3857 seemed to me like a shift and maybe a slight rotation, nothing too expensive. But it takes more than a second to compute which is probably around 90%< of the processing time the tileing requires.