maplibre-style-spec
maplibre-style-spec copied to clipboard
maplibre
Design Proposal: Contour Line Source from Raster DEM Tiles
Design Proposal: Contour Line Source from Raster DEM Tiles
Motivation
Give users a built-in way to render contour lines in maplibre from the same DEM tiles that are already used for terrain and hillshading, like this:
Proposed Change
Create a new contour source type in maplibre style spec that takes a raster-dem source as input and generates contour isolines as output that can be styled using line layers, for example:
sources: {
dem: {
type: "raster-dem",
encoding: "terrarium",
tiles: ["https://elevation-tiles-prod.s3.amazonaws.com/terrarium/{z}/{x}/{y}.png"],
maxzoom: 13,
tileSize: 256,
},
contours: {
type: "contour",
source: "dem",
unit: "feet" | "meters" | number, // default=meters, for custom unit use length of the unit like unit: 1.8288 for fathoms
// similar syntax to ["step", ["zoom"], ...] style expression
// to define contour interval by zoom level
intervals: [
200, // 200m interval <=z11
12, 100, // 100m interval at z12 and z13
14, 50, // 50m interval at z14
15, 20 // 20m interval >= z15
},
// put a "major=false/true" tag on every Nth line by zoom so styles
// can highlight major/minor lines differently
majorMultiplier: [
5, // every 5th line at < z14
14, 4, // every 4th line at z14
15, 5 // every 5th line for >= z15
],
// minzoom inferred from raster-dem source and maxzoom determined automatically by maplibre
// overzoom z10 tiles to generate z11 contour lines, z11 to make z12, etc...
overzoom: 1,
},
},
The generated isolines will have these attributes:
eleelevation above sea level in the unit specifiedintervalthe fixed interval between isolines at this zoom level in the unit specifiedmajortrue if this is a major isoline based on majorMultiplier at this zoom level, false otherwise
Layers can refer to the contours with source: contours but they can omit sourceLayer.
This offloads details about how to retrieve and parse DEM tiles to the DEM source definition, and gives style layers the flexibility to render any number of visible lines derived from that contour source.
I've already prototyped this in the maplibre-contour plugin which I'm using for contour lines on onthegomap.com. Here are some of the issues I had to work through to get these contours to look nice:
DEM "overzooming" (smoothing)
The contour lines look blocky when you zoom in much further past the maximum zoom for a DEM source, but they can look nice and smooth if you "overzoom" the DEM tiles by applying iterative bilinear interpolation before generating isolines. For example for onthegomap I use 512px z11 tiles, but overzoom the z11 tiles up to z15 so that the contour lines look smooth even at high zooms. This is why the proposal lets you specify a maxzoom that is higher than the maxzoom of the raster-dem source.
Also to generate smooth contour lines at the border between tiles, the algorithm needs to look at adjacent tiles. This means you need 9 DEM tiles to render a single contour tile. To mitigate this, the overzoom parameter lets you use overzoomed DEM tiles from a lower zoom level to generate contours at the current zoom level, for example overzoom=1 means use the top-left, top-right, botom-left, or bottom-right z10 tile to render a z11 contour line tile. This means you only need 4 DEM tiles to render a single contour tile:
Contour levels and units
The user needs to be able to choose what elevations to draw contour lines at, which changes by zoom level (rendering every contour would get too expensive at low zooms in hilly areas). They may also designate "major" and "minor" levels, for example generate thin contour lines every 200m but bold every 1000m. For now we will push this to layers that use the style, but in the future we can either add a major/minor designation to ticks, or pass-through the level and interval so styles can highlight every 5th or 10th line or something.
The unit attribute multiplies raw meter values by a certain amount to change the unit, for example unit=feet changes from meters to feet. When you click the distance indicator on onthegomap, it toggles between unit=meters and unit=feet. You could also set unit to a custom value for less common units like unit=1.8288 for fathoms.
Performance and Bundle Size
I've already implemented the smoothing logic and isoline generation in the maplibre-contour plugin so we would just need to bring that into maplibre-gl-js and port into the native projects. The overall plugin is 33kb (11kb gzipped) but most of that is replicating the web worker communication, cancelable message passing, and vector tile encoding that maplibre-gl-js already has. The actual smoothing+isoline business logic is only 3.7kb (1.6kb gzipped).
The isoline generation algorithm was derived from d3-contour but is much more efficient because it generates isolines in a single pass through the DEM tile instead of using a pass per contour level. For onthegomap users on a range of devices (mostly mobile phones) overzooming a 512px dem tile and generating isolines takes:
- <10ms 40% of the time
- >50ms 10% of the time
- >100ms 2% of the time
- >200ms 0.5% of the time
- >500ms 0.05% of the time
- >1s 0.006% of the time
API Modifications
This should only change the style spec, but shouldn't require any js or native API changes, unless we wanted to expose the default contour layer, elevation or level key as constants?
Migration Plan and Compatibility
This is new functionality, so no migration is necessary.
Rejected Alternatives
Build a plugin for this
I maintain the maplibre-contour plugin which already lets you do this by using the addProtocol integration, but it has a few downsides:
- It's an extra step to install: rendering contour lines is a common use-case that users should be able to do by default
- 90% of the plugin is duplicating things that maplibre already does like spawning a web worker, communicating with it using cancelable messages, and decoding DEM tiles. The actual code for computing the contours is a small fraction of the overall plugin.
- It has to do a wasteful extra step of encoding the result as vector tile bytes only for maplibre to decode immediately after in its own web workers (see https://github.com/onthegomap/maplibre-contour/blob/main/architecture.png)
- It can't make use of other registered maplibre request interceptors or protocols like DEM tiles served out of a pmtiles archive
- It doesn't work in maplibre-native
Pre-render contour lines
You can render contour line vector tiles ahead of time and serve those for the planet, this will save some browser CPU cycles but rendering them on the fly from DEM tiles has a few advantages:
- There are a lot of parameters you can tweak when generating contour lines from elevation data like units, thresholds, and smoothing parameters. Pre-generated contour vector tiles require 100+gb of storage for each variation you want to generate and host. Generating them on-the-fly in the browser gives infinite control over the variations you can use on a map from the same source of raw elevation data that maplibre uses to render terrain and hillshade.
- You're likely already downloading DEM tiles for hillshading and terrain, so this eliminates the extra bandwidth used to download those vector tiles.
Implement as a new layer type
We could implement this as a new layer type instead of a source type, but that would tightly couple the display parameters to the logic for how contour lines are generated, and potentially require us to generate the contours in multiple passes over the source DEM data. It seems cleaner to generate contour lines so you can generate as many layers as you want from them afterwards.
Take a DEM tile source URL as input
A contour layer could take as input tiles: ["server.com/{z}/{x}/{y}.png"], but there are a lot of different knobs to tune for how these are interpreted, so by depending on a DEM source we re-use the DEM source control all of those parameters.
This API is what I landed on for configuring the maplibre-contour plugin, but there are a few spots I could go either way on:
- instead of
multiplier=1multiplier=3.28084we could useunit=meters/unit=feet- it's less flexible but easier to use - do we need a way to set a minimum elevation for the isolines? You may not want bathymetry but there are plenty of places on land that are below sea level
- should the "major" / "minor" isoline decision happen in style layers? It seemed a little more natural alongside where we define the contour line levels in the source config
- do
elevationKeycontourLayerandlevelKeyneed to be configurable?
This is a great write up! Thanks! Regarding using a dem source, there are some issues with which tiles to fetch and use, so it might prove better to copy some of the raster dem source definitions into this source, there aren't a lot of parameters, so duplication it might not be such a bad idea.
I agree about the units, I think feet and meters are more readable and are the only options available for the scale control. I also think the major/minor should be a layer configuration, hopefully with some expression logic and avoid placing this configuration in the source itself.
Other than that, I think this should get it as it will give maplibre a competitive advantage over other libraries.
I'll bring it up in the next monthly web meeting, feel free to participate.
Thanks @HarelM!
Regarding using a dem source, there are some issues with which tiles to fetch and use, so it might prove better to copy some of the raster dem source definitions into this source
What are the issues? One of the reasons for moving out of the plugin into maplibre would be to make use of the shared DEM tile cache between sources.
I also think the major/minor should be a layer configuration
Just to illustrate the difference, my config from the demo would look like this with where the source sets a level property, and layers use it when styling the lines:
{
sources: {
contour_feet: {
type: "contour",
source: "dem",
maxzoom: 16,
multiplier: 3.28084,
thresholds: {
11: [200, 1000],
12: [100, 500],
13: [100, 500],
14: [50, 200],
15: [20, 100],
}
}
},
// ...
layers: [
{
id: "contours",
type: "line",
source: "contour_feet",
"source-layer": "contours",
paint: {
"line-color": "rgba(0,0,0, 50%)",
"line-width": ["match", ["get", "level"], 1, 1, 0.5], // make major contours bolder
},
layout: {
"line-join": "round",
},
},
{
id: "contour-text",
type: "symbol",
source: "contour_feet",
"source-layer": "contours",
filter: [">", ["get", "level"], 0], // only put labels on major contours
layout: {
"symbol-placement": "line",
"text-size": 10,
"text-field": ["number-format", ["get", "ele"], {}],
"text-font": ["Noto Sans Bold"],
},
},
]
}
but it would look like this if the major/minor determination is entirely within the layer definition (derived from elevation):
{
sources: {
contour_feet: {
type: "contour",
source: "dem",
maxzoom: 16,
multiplier: 3.28084,
thresholds: {
11: 200,
12: 100,
13: 100,
14: 50,
15: 20
}
}
},
// ...
layers: [
{
id: "contours",
type: "line",
source: "contour_feet",
"source-layer": "contours",
paint: {
"line-color": "rgba(0,0,0, 50%)",
// thicker major lines
"line-width": [
"step",
["zoom"],
["match", ["%", ["get", "ele"], 1000], 0, 1, 0.5],
12,
["match", ["%", ["get", "ele"], 500], 0, 1, 0.5],
14,
["match", ["%", ["get", "ele"], 200], 0, 1, 0.5],
15,
["match", ["%", ["get", "ele"], 100], 0, 1, 0.5],
]
}
},
{
id: "contour-text",
type: "symbol",
source: "contour_feet",
"source-layer": "contours",
// only put labels on major lines
filter: [
"step",
["zoom"],
["==", ["%", ["get", "ele"], 1000], 0],
12,
["==", ["%", ["get", "ele"], 500], 0],
14,
["==", ["%", ["get", "ele"], 200], 0],
15,
["==", ["%", ["get", "ele"], 100], 0],
],
layout: {
"symbol-placement": "line",
"text-size": 10,
"text-field": ["number-format", ["get", "ele"], {}],
"text-font": ["Noto Sans Bold"],
},
},
]
}
The second one makes layer definitions repeat a lot, and also requires keeping the major/minor level logic in sync with the level thresholds by zoom from the source. For example if you have 50m lines with 250m major lines, but you change the interval to 100m then the 250m/750m major lines will never show up.
What are the issues?
In terrain it is advised to use a different source instead of using the one for hillshade as the logic of which tile to fetch is a bit different due to how the terrain works.
Regarding the source and layer coupling, it is the same for other vector sources as well, if you change the definition of a layer in the source you'll need to adapt the style. Although you can place certain features in certain zoom levels in the source, so I'm not sure what's "cleaner".
I'll post it on slack to hopefully get more feedback.
In terrain it is advised to use a different source instead of using the one for hillshade as the logic of which tile to fetch is a bit different due to how the terrain works.
Is this because we want to switch what zoom level DEM tile we use based on the current zoom differently with terrain vs. hillshade? It will be slightly different for contour lines too if you set the overzoom parameter, but it seems like they should still all be able to pull from a shared tile cache and share the decoding config?
I'm not entirely sure about the details, but there is an opposite direction between cache and source I think, source cache is managing a source instead of the other way around.
instead of multiplier=1 multiplier=3.28084 we could use unit=meters / unit=feet - it's less flexible but easier to use
If you’re considering bathymetry, then fathoms might also be relevant. Apart from that, would you care for a map contoured in smoots? 😎
Wow, this looks great to me. Since in most cases we have hillshading /terrain anyways and thus the bytes are already going through the tubes anyways...
Ship it.
What am I missing?
I'm not entirely sure about the details, but there is an opposite direction between cache and source I think, source cache is managing a source instead of the other way around.
OK, this seems like a gap in the current implementation that would be unfortunate to leak into the style spec. It seems like terrain/hillshading/contours reading from the same tiles should be able share a source definition... I'll understand the limitations better when I get into the implementation, but I think it would be good to try to shoot for something consistent with how terrain lets you refer to a raster source by ID, then adjust if that looks like it's causing more problems than it's worth?
Sure, implementation details of the web library shouldn't affect the spec. There's no cross reference between sources in the spec right now, but I'm not saying there shouldn't be. I'm fine either way.
If you’re considering bathymetry, then fathoms might also be relevant. Apart from that, would you care for a map contoured in smoots? 😎
Another option here would be something like:
unit: 'feet' | 'meters' | number
So we get the convenience of being able to specify the most common unit by name, but flexibility of being able to use unit=1.7018 for smoots?
What is the performance of downloading 1 PNG for Hillshade and 1 PNG for Countour versus downloading a DEM and the device calculating/drawing?
The calculations are different, and the browser's cache can help in case it's the same tiles, but this proposal is for native as well so let's try and focus on the style spec changes. The implementation details can be discussed in the relevant repo in a PR or an issue.
For my setup currently, it's around:
- tile fetch p50=90ms p90=300ms p99=1s
- 512px contour generation p50=17ms p90=50ms p99=80ms
Although that includes browser cache, and the extra step of encoding the result as vector tile bytes, which won't be necessary any more. 256px tiles should also take 1/4 the time.
Vector tile decoding and processing also takes a nontrivial amount of time, often longer than the network request to fetch the tile on web.
@msbarry I think it is a good idea to add the contour lines plugin's functionality directly to MapLibre GL JS, because
- contour lines are a widespread use case
- a lot of code duplication can be avoided
- size impact and maintenance impact on GL JS are minimal
Regarding the implementation in the style spec, I found it sometimes confusing how the major and minor lines were defined in the plugin with the mapping from thresholds to levels. Maybe we could rename thresholds to levels?
We discussed at the monthly steering committee meeting, it sounds like there's general consensus to move forward on this. I'll try to simplify the source definition a bit and see what other similar tools do to specify contour levels and major/minor ticks for comparison.
My open questions:
- do the overzoom/maxzoom parameters make sense?
- are the attributes that control the generated vector feature attributes necessary? (
elevationKey/levelKey/contourLayer) If we exclude them then there will just be some magic/default layer and tag names that layer styles need to use to render the lines.
I think a user using this source should be able to tweak these parameters in theory, having said that, I think we can start off by using some hardcoded "magic" strings in the original definition to avoid the extra complexity, if there's a need to tweak them we can add support for it later on in the next versions. Another option is to have them in the spec and have default values, this way there's no need to specify them explicitly if one doesn't need to change the defaults. Either options work for me, I personally prefer simplicity and a two phase manner seems like a good approach (i.e. if no one needs to actually tweak this and the defaults are good, then why complicating, right?).
Another input from the meeting yesterday was the zoom definitions, mush like the above, I think we can start of by allowing the logic in the layers and then adding extra complexity if needed in the future.
Excited to see this coming to MapLibre!
The multiplier attribute multiplies raw meter values by a certain amount to change the unit, for example multiplier=3.28084 changes from meters to feet. When you click the distance indicator on onthegomap, it toggles between multiplier=1 and multiplier=3.28084.
It may be advisable (on the implementation side, so this is a note for future reference) to include a few common constants we document for people, so we don't have to look up meters to feet indefinitely in the future. 😄
About the multiplier, see @msbarry 's comment which I think is a good solution for this: https://github.com/maplibre/maplibre-style-spec/issues/583#issuecomment-2029497763
I think a user using this source should be able to tweak these parameters in theory, having said that, I think we can start off by using some hardcoded "magic" strings in the original definition to avoid the extra complexity, if there's a need to tweak them we can add support for it later on in the next versions. Another option is to have them in the spec and have default values, this way there's no need to specify them explicitly if one doesn't need to change the defaults. Either options work for me, I personally prefer simplicity and a two phase manner seems like a good approach (i.e. if no one needs to actually tweak this and the defaults are good, then why complication, right?).
OK great, I updated the definition to include unit = "feet" | "meters" | number and removed elevationKey/levelKey/contourLayer. I kept overzoom, as I think it's important to be able to change since it drives the smoothness of generated lines and tile over-fetching. I could see defaulting it to 1 but still seems like it should be configurable.
It may be advisable (on the implementation side, so this is a note for future reference) to include a few common constants we document for people, so we don't have to look up meters to feet indefinitely in the future. 😄
@lseelenbinder are there any other unit constants you think we should include besides feet and meters to start?
Another input from the meeting yesterday was the zoom definitions, mush like the above, I think we can start of by allowing the logic in the layers and then adding extra complexity if needed in the future.
I think we can omit the major/minor designation for now, but we do need the source to specify at what interval contour lines should be generated. The contour generation performance is proportional to the number of lines that end up on the tile, so if we pick the lowest common denominator then low zoom tiles in hilly areas will slow to a crawl.
For comparison, gdal-contour lets you specify:
-a <name>Provides a name for the attribute in which to put the elevation. If not provided no elevation attribute is attached. Ignored in polygonal contouring (-p) mode.-i <interval>Elevation interval between contours.-off <offset>Offset from zero relative to which to interpret intervals.-fl <level>Name one or more "fixed levels" to extract.-e <base>Generate levels on an exponential scale: base ^ k, for k an integer.-nln <name>Provide a name for the output vector layer. Defaults to "contour".
It also lets you generate polygons (isobands instead of isolines) which I'd probably try to avoid in maplibre unless there's a strong use-case?
d3-contour requires you specify a list of "thresholds" (where my original terminology came from).
The most flexible version I could see ending up with might look like:
levels: {
[zoom: number]: interval | [list, of, thresholds] | { min, max, offset, exponent, ...}
}
but we could start by just supporting a single number then add more complex variations based on feedback?
Another option instead of a map from zoom level to interval would be to allow a zoom-based expression, like:
levels: [
"step", ["zoom"],
200,
11, 100,
14, 50,
15, 20
]
WDYT @HarelM ? I'm not sure if it will cause trouble running a maplibre expression outside the context of an individual feature?
Yea, I wouldn't want to try and use the expression engine in a source property, sounds like asking for trouble.
I get how generating "too many" contours for a specific zoom can be problematic.
I like the term intervals more than levels.
In theory one could split the work between two or more sources to reflect min and max zoom for each different interval, but I'm not sure it's any better...
It also lets you generate polygons (isobands instead of isolines) which I'd probably try to avoid in maplibre unless there's a strong use-case?
These can create really cool effects (e.g., bathymetry layering), but I don't think it's a problem to leave it off the spec because if we actually have a strong use-case, it's easy enough to add later.
I also prefer intervals to levels because
-fl <level>Name one or more "fixed levels" to extract.
is actually something different than intervals, IIUC. It's more about specific set of levels (e.g., only 0, 250, 1000), so we could potentially add that type of functionality in the future.
These can create really cool effects (e.g., bathymetry layering), but I don't think it's a problem to leave it off the spec because if we actually have a strong use-case, it's easy enough to add later.
Ah sorry to clarify I avoided the isobands because I'm not sure if there's a way to do it efficiently enough in the client yet. D3-contour creates all of the contour polygons then tests which ones contain the others to assign shells and holes, which can be expensive compared to naively creating the lines and not having to worry about closing them. There's probably a way to do it with minimal performance impact, I just haven't fully thought through that yet.
I like the term intervals more than levels.
👍 changed the definition to intervals. @HarelM that makes sense about not wanting to use expressions directly, but people have already learned the quirks of the step expression, instead of adding a new { [zoom]: interval } format that people need to learn what happens when a level is missing, what if we re-use the step syntax without it actually being an expression?
intervals: [
200, // value when < z11
11, 100, // >= z11
14, 50, // >= z14
15, 20 // >= z15
]
Then people could shorten to
intervals: [200]
or
intervals: 200
if they want the same contours at every zoom?
I see what you mean with the fact that one needs to define what happens for every zoom level, and a "dictionary" style syntax would be very verbose if you have 10 zoom levels for example. The above syntax let you define what happens for a range of zoom levels. I don't like using arrays this way and it's super hard to validate, debug, etc, but since, as you said exist in this library in other places in the style, the users may know how to deal with it. I think it makes sense. I would love to hear other opinions. The initial post has the most up-to-date spec that is propose, and at it current state I think it should be approved. Let's give people one more week to look at this and comment.
I would prefer a more explicit definition like for example this one:
intervals: [
0, 10, 200, // if 0 <= z <= 10, use 200
11, 13, 100, // if 11 <= z <= 13, use 100
14, 15, 50, // if 14 <= z <= 15, use 50
15, 21, 20 // if 15 <= z <= 21, use 20
]
This has on the other hand the downside that the max zoom has to be specified.
EDIT: I might be wrong. The syntax that you suggested @msbarry seems closer to the already existing step syntax...
what if we re-use the step syntax without it actually being an expression?
I appreciate how this approach would leave open the door to introducing support for more flexibility in the future without requiring a brand-new syntax or some sort of deprecation dance. There’s already plenty of precedent in the style specification and API for allowing an expression but not any arbitrary expression. On the other hand, if someone sees a part of the style specification that allows a step expression, they’ll likely be surprised at first if interpolate is unsupported. Regardless, make sure to avoid designing anything that could be confused with zoom functions, which have lingered on long past their expiration date.
@1ec5 I was thinking that it would just include the values from within a ["step", ["zoom"], ...] expression, like just interval: [100, 11, 50] to do 100 <z11 then 50 at z12 and above. That makes it so there's no question about using a different expression type, but also less clear of an upgrade path to more complex expressions.
I'd like to suggest a different approach to the major/minor question. This has been implemented in the DEM-based (pre-generated) contour lines of Israel Hiking Map.
The contour features have an additional rank attribute that has one of the following values: 1, 2, 5, and 10.
[Edited:] A rank = n is assigned to every n'th contour line:
- A
rank=10is assigned to every 10'th contour line. - Otherwise, a
rank = 5is assigned to every 5'th contour line - Otherwise, a
rank = 2is assigned to every 2'nd contour line - Otherwise, a
rank = 1is assigned to all remaining contour lines
This allows the style to decide about the density various style elements across the contour lines.
Below is the paint part of the IHM style use for the lines of all contours in all zoom levels that would paint a thicker line every 5 contour lines:
"paint": {
"line-color": "rgb(161, 127, 86)",
"line-width": [
"match",
["get", "rank"],
[1, 2],
1,
2
]
}
Similarly, this is the filter part of the style that places the elevation labels also on every 5'th line
"filter": ["!in", "rank", 1, 2]
For example, these are some of the elevations that get each rank, according to the interval used in a given zoom level:
| interval | rank: 1 | rank: 2 | rank: 5 | rank: 10 |
|---|---|---|---|---|
| 10 | 10, 30, 570, ... | 20, 40, 680, ... | 50, 150, 750, ... | 0, 100, 200, 800, ... |
| 20 | 20, 40, 520, ... | 40, 60, 680, ... | 100, 300, 700, ... | 0, 200, 400, 1200, ... |
| 50 | 50, 150, 550, ... | 100, 200, 700, ... | 250, 750, 2250, ... | 0, 500, 1000, 3500, ... |
| 100 | 100, 300, 5700, ... | 200, 400, 6800, ... | 500, 1500, 7500, ... | 0, 1000, 2000, 8000, ... |
