mapbox-gl-js
mapbox-gl-js copied to clipboard
mapbox
GeoJSON polygon rendering is unreliable (master ticket)
This is an umbrella ticket for various reports of occasional artifacts when rendering polygons with GeoJSON source, with a summary of the issue and our road to a solution.
Previous tickets: #12356, #10768, #10592, #10299, #10106, #9981, #9913, #9761, #9441, #9072, #7857, #7663, #7433, #7228, #6383, #6313, #6069, #3545, #3080. And related ones: #13147 #12903 #7748, #7233, #5265, #4962, #3032, #2975, #2696
To be able to render polygons in WebGL, we have to turn them into a set of triangles — this is done by the earcut library. It is very fast, but has one serious limitation — it can't handle non-simple polygons, such as those with self-intersections, intersecting rings, or holes outside of the outer ring. This is itself a constant point of confusion (as suggested by related tickets above), especially given that Canvas and SVG don't have such problems (browser rendering engines handle degeneracies internally). But at least in the invalid polygon case, we can offer a workaround — fixing the input polygons before feeding them to GL JS.
What makes matters worse is that polygon geometries have to be processed before being rendered — specifically, they have to be sliced into tiles, with shapes converted into an integer tile coordinate system and simplified for every zoom level — performed by the geojson-vt. This process alters original geometry in subtle ways that can lead fully valid simple polygons to become invalid on certain zoom levels — in particular, introduce self-intersections. This in turn, although rarely, triggers rendering artifacts in GL JS for which we don't have any workarounds to offer.
This is a fundamental problem that I've been meaning to address for a long time, but it is notoriously difficult to solve algorithmically. The only attainable solution I see is fixing polygons at runtime after processing on the client. This is what we already do in GL Native with wagyu. So we have two options:
- Port Wagyu to JavaScript. This C++ library is 5000 lines of very complex code, so the port would be very difficult, potentially add a significant overhead to the GL JS bundle size, without any guarantee that the JS port will be performant enough to handle the issue.
- Compile Wagyu with Emscripten into a browser version (either JS or potentially WebAssembly in future) and use it is a kind of a drop-in plugin for situations when the problem arises, similar to how we solved RTL text rendering. This is our last resort solution, because the library still won't work by default, but it may work well enough and at last provide a practical workaround for some relief.
- Come up with a new, lightweight, JS-centric approach. This is what I have attempted multiple times throughout the years in the polysnap project, and the approach is very promising. I feel like I came very close to a working solution recently (although the WIP code is not yet pushed to the repo), but still need more time to tackle this — hopefully in the nearest months.
Also note that solving the issue for the valid polygons use case (where they turn invalid later in the GL JS pipeline) will automatically solve it for other use cases such as invalid input polygons, make our API easier to use.
I'll provide updates to this ticket when the matters progress — stay tuned. Meanwhile, I'm closing the open tickets among the linked ones to centralize the discussion.
A few other possiblities - unsure how each of them would work:
if geojson-vt is causing segmentation issues and if the errors introduced by geojson-vt are predictable enough, could we not fix that issue at source in geojson-vt?
A further suggestion as a hotfix for the current scenario - could we set a flag to disable this tiling for simple shapes that are causing defects, and just display the shape at full detail over all zoom levels? I.E. bypass geojson-vt altogether.
if geojson-vt is causing segmentation issues and if the errors introduced by geojson-vt are predictable enough, could we not fix that issue at source in geojson-vt?
They aren't predictable — basically, any rounding of a coordinate to integer grid can introduce a self-intersection, even if you set simplification tolerance to 0. Also, there are no known fast simplification algorithms that preserve topology and are guaranteed to avoid self-intersections.
A further suggestion as a hotfix for the current scenario - could we set a flag to disable this tiling for simple shapes that are causing defects, and just display the shape at full detail over all zoom levels? I.E. bypass geojson-vt altogether.
No, because the whole architecture of Mapbox GL relies on the data being tiled and converted to integer tile coordinates.
@mourner et al - running into this issue in GL JS 0.46.0 and higher.
https://bl.ocks.org/samfader/raw/0c68b85662ecb933bc5cbcfb7a673cfc/ - uses GL JS 0.46.0
https://bl.ocks.org/samfader/e6806f9961daf7aeb3a5123610f207f2 - uses GL JS 0.45.0
The polygons look fine in geojson.io and in GL JS 0.45.0 and lower, and also when created as tilesets via Studio, but if added in GL JS in a version equal to or higher than 0.46.0, the polygons start to get randomly simplified/cut off. Do you think this is related to this ticket, or worth a separate one?
I have another example of this issue on mapbox-gl-js 0.46.0+ https://codepen.io/anisart/pen/wYpwpb - geometry breaks on 5+ zoom level.
But I don't use non-simple polygons. I need draw squares only, when every square is equal to tile of some given zoom level. Is there any solution for this case? Or should I use the version 0.45.0 for now?
@anisart no, that looks like a different issue — might be a regression. Let me take a look.
#7748 is also likely related to this issue. Setting maxzoom: 18 in my geojson source options eliminates the 'overzooming' artifacts in my example on that issue.
Hi,
I encountered this problem but have managed to resolve most of the artefacts by using fill-antialias: false.
Here's an example of a polygon layer with geoJSON:
const layer_low ={"id": 'poly_lo', "type": "fill", "source": 'data_positions', 'paint':{'fill-outline-color': '#ff0', 'fill-color': '#ff0', "fill-opacity": 0.3}, filter: ['==', 'conf', 'Low']}
I have two other layers with similar definitions. This results in the following display:
But, if I change my definition to this:
const layer_low ={"id": 'poly_lo', "type": "fill", "source": 'data_positions', 'paint':{'fill-outline-color': '#ff0', 'fill-color': '#ff0', "fill-opacity": 0.3, 'fill-antialias': false}, filter: ['==', 'conf', 'Low']}
Then the map output is as follows:
Much better! Thought I'd mention it here in case it helps anyone else.
I assume this is related to this issue? I have a very simple polygon in my tileset that I am rendering through MapboxGL JS
Here are some screenshots showing very glitchy behaviour:
Zoom level 14.97:
Zoom level 15.45:
Zoom level 16:
Zoom level 16.95:
And finally it looks ok at Zoom level 17:
@vomc no, this one looks more like low maxzoom of a GeoJSON source — set it to a higher value. Although I can't say for sure without a reproducible live test case.
@mourner OK yeah I tested it in isolation (https://jsfiddle.net/r15bcn7q/) and this issue is not happening when I just add a geoJSON poly to MapboxGL. In the example for which I posted screenshots above, the geoJSON polygons are coming out of a Mapbox Studio created tileset. I have tried adding a maxzoom property to the source but that doesn't seem to affect anything...
I wrote a polygon drawing tool for my company's product, and thought that I had a bug until I noticed that this affects mapbox-gl-draw as well. Here's a screenshot directly from the docs (panned over water for visual contrast):
Other than that, nothing new to add, but in terms of priority let's all keep in mind that while we often refer to self-intersecting polygons as "invalid" in these issues, they are valid GeoJSON.
Also, in case this helps any googlers out there, here's a snippet that detects this issue and attempts to fix it by re-ordering the vertices in counter-clockwise order. While this does technically solve the self-intersection by creating a simple polygon using the given vertices, sometimes that isn't the polygon you wanted, making it unsuitable for a drawing application.
import kinks from '@turf/kinks';
if (kinks(feature).features.length) {
const coordinates = feature.geometry.coordinates[0];
coordinates.pop(); // remove the closing lngLat
// Sort from top to bottom.
coordinates.sort((a, b) => a[1] - b[1]);
// Get center y
const cy = (coordinates[0][1] + coordinates[coordinates.length - 1][1]) / 2;
// Sort from right to left
coordinates.sort((a, b) => b[0] - a[0]);
// Get center x
const cx = (coordinates[0][0] + coordinates[coordinates.length - 1][0]) / 2;
// Center lngLat
const center = [cx, cy];
let startAng;
// Precalculate the angles
coordinates.forEach(lngLat => {
let angle = Math.atan2(lngLat[1] - center[1], lngLat[0] - center[0]);
if (!startAng) {
startAng = angle;
// ensure that all lngLats are clockwise of the start lngLat
} else if (angle < startAng) {
angle += Math.PI * 2;
}
lngLat.angle = angle; // add the angle to the lngLat
});
coordinates.sort((a, b) => b.angle - a.angle);
coordinates.push(coordinates[0]); // Close the polygon
return feature;
}
I wonder if we can copy some ideas from the lyon project, or use its wasm build. The blog post sounds very similar:
Some of these reports were easy to address, some were very hard, and it almost always boiled down to precision loss introduced by arithmetic operations performed when detecting and handling self-intersecting geometry. ... Long story short, there were some rare but very hard to fix bugs that required rethinking core parts of the algorithm.
and
Fixed-point numbers, while providing a somewhat consistent precision loss that I had an easier time wrapping my head around, still lost precision and didn't solve the root of the issue. They helped with the easy problems and got in the way of fixing the hardest ones.
I am facing the same issues currently at any zoom level
Desired
Actual
I'm just dumping some random thoughts and findings about this type of issue from another perspective. Our company has an own 3D rendering engine (called Deep Map), and we've been facing similar issues in the past. Our main focus is on indoor visualization, thus we also have to deal with different projections (usually a fitting UTM projection - web mercator is just a no-no as it looks bad for indoor spaces and building outlines in many parts of the world the farther away you are from the equator).
We've been investigating into using MVT in the past, but found mainly two issues (for our use case): The storage is bound to a projection (would need different tile sets per projection and maintaining those), and reprojecting tile data isn't easy and introduces artifacts.
At the moment we have an own format called HSG ("Heidelberg Sinusoidal Grid"), a binary format using FlatBuffers, storing full geometry data in geographic coordinates. The transformation to the target visualization projection (usually UTM) is done at runtime. Geometries overlapping tiles are stored in one tile only: tiles have kind of relationship info (geometries overlapping tiles result in tile dependency info to load also out-of-bounds tiles containing those required geometries).
This served us well for a few years, but this approach has also shortcomings: Large geometries have to be split and treated in a different way, and perspective viewing is still not perfect using a classical tile scheme. Also, a 3D map is better served with a 3D solution rather than working around certain types of issues when using a 2D grid (tile pyramid).
I think we're coming from a different angle than Mapbox in our requirement regarding projection support from the beginning. But it looks like we've on both ends spent significant effort for dealing with tile processing, map projections, and geometry degeneracies.
If I'd be tasked with doing the next generation storage engine, it would roughly work like this:
-
Go full 3D in visualization (show a globe per default): There is perspective viewing support, 3D models. And there are legacy API methods (in case of Mapbox) which don't fit perfectly anymore (like zooming to a 2D bounding box in 3D)
-
Go full 3D with the storage engine: 3D data doesn't fit in a 2D grid well. And also, 2D map projection requirements usually mainly come from people requiring more precise visualization, e.g. nobody would ask for Albers projection within Google Earth - you'd rather convert your data from Albers projection to EPSG:4326 and render it away on a globe and be happy with the result. For the folks requiring a 2D map as you have it today: It should be quite easily possible to implement switching between 3D visualization and Web Mercator in the GL shader. If it's done at that end, no degeneracies are to be expected (regarding polygon validity). I'd also scrap all previous efforts in writing an own grid, and simply chose Google's S2 library for the global 3D grid.
-
Using full geometry data adds a little complexity to tile loading, but it proved to be working well and avoids various edge cases. But I'd treat very large geometries (like coast lines) in a MVT-like way as a fallback.
I was having an issue rendering a large, self interesecting polygon. In my case, I was able to fix this by breaking up the large, self-intersecting polygon into seperate non-intersecting polygons and then adding them to a single layer as part of a multi-polygon. Obviously this will only work if you can determine where the intersection are (which I did not need to do because it is defined by real world dynamics that cannot intersect in a small epoch so I could just group polygons by time).
I have another example of this issue on mapbox-gl-js 0.46.0+ https://codepen.io/anisart/pen/wYpwpb - geometry breaks on 5+ zoom level.
But I don't use non-simple polygons. I need draw squares only, when every square is equal to tile of some given zoom level. Is there any solution for this case? Or should I use the version 0.45.0 for now?
https://codepen.io/paullobo/pen/OJXMvaj
If you complete the polygon with starting array then it shouldn't cause the clipping issue
I have an example using a polygon with holes where artifacts appear based on the zoom level.
https://user-images.githubusercontent.com/4028068/104392905-be8dde80-5508-11eb-9756-59a70573addc.mp4
I am experiencing a very similar issue. In our case we are taking a geoJson object, using turf.difference to create a mask shape. When we add the layer to the map it looks correct at a zoom level of 10. However, when we zoom in further around zoom level 11 and 12, we get strange shapes appearing. This typically only happens with more complicated shapes with holes. To avoid complications with Turf while dealing with complicated shapes, we are running calculations to separate interior shapes and add them as separate layers - this is working fine. However, the issue is still present when only creating a mask with the main shape and exterior shapes.
Correct Geometry View:
Incorrect View at zoom level 11:
Is there any way to fix this issue currently?
The only 100% fix I know of is to not use webgl for rendering, which means canvas via leaflet or openlayers. For my use case I found the errors tolerable, but never found a solution. I did find less errors by pushing geojson to mapbox GL JS and not trying to push custom vector tiles.
On Fri, Sep 10, 2021, 10:37 AM Harry Borrelli @.***> wrote:
I am experiencing a very similar issue. In our case we are taking a geoJson object, using turf.difference to create a mask shape. When we add the layer to the map it looks correct at a zoom level of 10. However, when we zoom in further around zoom level 11 and 12, we get strange shapes appearing. This typically only happens with more complicated shapes with holes. To avoid complications with Turf while dealing with complicated shapes, we are running calculations to separate interior shapes and add them as separate layers - this is working fine. However, the issue is still present when only creating a mask with the main shape and exterior shapes.
Correct Geometry View: [image: correct-geometry-view] https://user-images.githubusercontent.com/15935329/132870362-64eb73e5-7748-43b5-b140-5151fa016a5e.png
Incorrect View at zoom level 11: [image: screenshot-of-issue] https://user-images.githubusercontent.com/15935329/132870466-cdd843e8-a5a1-431c-a183-26d204c3ff17.png
Is there any way to fix this issue currently?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mapbox/mapbox-gl-js/issues/7023#issuecomment-916954424, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ2HUJEJN2RMN5YECFZFU3UBIJ3ZANCNFSM4FMBMPOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
@hborrelli1 currently the only 100% reliable workaround is to upload and use it as a vector tile dataset instead of GeoJSON — Mapbox vector tile server has advanced processing that cleans up all topologic issues with the polygons in each tile, so they're pretty much guaranteed to render correctly. The difficulty is in bringing this processing to the client, which prompted the ticket.
Interesting. I'm not supporting the maps where this was an issue, but it's nice to know it's at least partially fixed. 👍
On Fri, Sep 10, 2021, 2:56 PM Vladimir Agafonkin @.***> wrote:
@hborrelli1 https://github.com/hborrelli1 currently the only 100% reliable workaround is to upload and use it as a vector tile dataset instead of GeoJSON — Mapbox vector tile server has advanced processing that cleans up all topologic issues with the polygons in each tile, so they're pretty much guaranteed to render correctly. The difficulty is in bringing this processing to the client, which prompted the ticket.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mapbox/mapbox-gl-js/issues/7023#issuecomment-917136567, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ2HUN72R4QDWI7EJRQAOTUBJIEJANCNFSM4FMBMPOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
@mourner - the problem with using vector tiles for the case that @hborrelli1 has described in the fact that we are not able to achieve the same result and I'll explain why. We have a Postgres table with different types(city, county, municipality, zip code, etc) of places in the USA.
For a line layer (red line on @hborrelli1's screenshot) it's quite simple (let's ignore now there are some artifacts on the map):
But how to gray out the rest territory? The following filter will not give us a needed result as there are other places located in the same place where feature ca-state-place-placercountyunincorporated is located. It will be annoying to exclude all of them by id's and not necessary they fit into ca-state-place-placercountyunincorporated shape boundary.
"filter": [
"all",
[
"!=",
"id",
"ca-state-place-placercountyunincorporated"
]
],
Do you know any examples of how to grey our territory around a feature according to my example?
For what it's worth, I wrote a little JS module to deal with non-simple polygons (back in the days when I was trying to write a buffer algorithm using JS for the Turf project). Just in case it could be of help with any clean-up process. Good luck!
I'm not sure if my idea is valid (please correct me) but I've tried generating GeoJSON feature for polygon by first creating a polygon in Leaflet and converting it to GeoJSON using toGeoJSON() method. Then I used that generated polygon feature in Mapbox application.
Any update on this?. I have tried with the leaflet library and geojson.io and they show me well but in mapbox v2.11.0 it is not rendering well
I still have this problem with the latest version (v3.0.0-Bet.4), I wonder when it will be fixed?
I have a problem that might be related which I thought had to do with self-overlapping MultiPolygons:
https://github.com/mapbox/mapbox-gl-js/assets/5530960/9250d1f0-992a-4169-9ca8-c9c3e3e42aa0
On closer inspection, this happens because of the size and positioning of our layers (we cover the entire world except a region of interest such as a country or province by our own coordinates).
Are there any heuristics to make this happen less often? This is a massive visual glitch for us, we use this layer to show customers where they can use our product.
