Add "raster-colorize" property

[cebartling]

Motivation

Mapbox GL JS uses raster and vector tilesets as core pieces of making maps visible in the browser and relies heavily on both raster and vector tilesets to keep their maps fast and efficient. Our company will mostly use raster-based tilesets. Raster tilesets are created when raster images, in TIFF and GeoTIFF format, are uploaded and processed by Mapbox Studio. Our company currently uploads various types of mosaic raster images into Mapbox Studio for creating raster tilesets that can be layered onto maps rendered in the Mapbox GL JS library. Mapbox itself provides their satellite map imagery as a raster tileset.

Our company has some more complex requirements where certain raster tilesets will need to be colorized on the client-side using a data-driven approach. This colorization process needs to be performed client-side as the colorization scheme is manipulated by our users in our web UI. Therefore, our developers will need to be able to hook into the rendering pipeline of the Mapbox GL JS library and provide colorization information to the fragment shader used to transform raster image pixel color.

Design Alternatives

This need for client-side, data-driven rendering hooks has been cited by other customers. Ubilabs wrote a blog post about how they built this feature into a forked version of Mapbox GL JS. In that writeup, they build a colorization lookup texture in JavaScript that maps values in the range of 0 to 255 to a unique color. This colorization lookup texture is then passed to a custom fragment shader that transforms each pixel value of an input raster tile to a color mapped by the generated colorization lookup texture. The input raster tile is grayscale, thus locking all color channels, so only the red channel is used to determine pixel value. Those pixel values can range from 0 to 255.

This is exactly the solution our company needs for our colorization of mosaic images layered on the map. The downside of the Ubilabs solution is that they forked Mapbox GL JS to get their solution to work. Our company would like to avoid having to fork the Mapbox GL JS module to get this functionality.

[lucaswoj]

@cebartling Could you clarify what you mean by "client-side, data-driven rendering hooks" with an API mock up? Would a data-driven raster-colorize property meet your use case?

[cebartling]

@lucaswoj This blog post https://www.mapbox.com/blog/tilemill-raster-colorizer/ is very similar to what I would like to do with the Mapbox GL JS component. I like TileMill's raster-colorizer style API-it fits very well with what we're doing:

{
  raster-opacity:1;
  raster-colorizer-default-mode: linear;
  raster-colorizer-default-color: transparent;
  raster-colorizer-stops:
    stop(0,#47443e)
    stop(50, #77654a)
    stop(100, rgb(85,107,50))
    stop(200, rgb(187, 187, 120))
    stop(255, rgb(217,222,170));
}

A raster-colorize property on the map layer could be set with this style to colorize the raster image tiles. Setting the property would trigger a re-colorization and render of the raster image. Does that make sense?

[averas]

Interesting! Openlayers 3 has similar hooks to perform pixelwise operations on raster sources pre-rendering:

https://openlayers.org/en/latest/examples/raster.html https://openlayers.org/en/latest/examples/shaded-relief.html https://openlayers.org/en/latest/examples/region-growing.html

[lucaswoj]

That makes sense @cebartling. Thanks! I'm retitling this issue to focus discussion on the idea of a "raster-colorize" style spec property.

This might dovetail nicely with https://github.com/mapbox/mapbox-gl-js/issues/3605.

[cebartling]

@lucaswoj Great! Thanks for engaging on the enhancement request so quickly.

[andrewharvey]

The style spec issue for this is at https://github.com/mapbox/mapbox-gl-style-spec/issues/476

[Scarysize]

Nice to see a growing interest on the subject. Of course we (Ubilabs) would love to have this feature in mapbox-gl-js; we would still be maintaining our own fork (mainly for data-driven sdf icons), but this would lead to one thing less to worry about. I'm excited for your implementation! If you have questions about ours, we would love to help!

[stevage]

This would be really cool. I'm really hoping to port cycletour.org from Tilemill to Mapbox-GL-JS one of these days, and this (and slope shading) is pretty much the big blocker. There's really no equivalent in GL-JS. (There are equivalents for slope shading, but they're a fairly poor substitute IMHO.)

With this, some generous company could host elevation raster tiles for the whole world, and everyone could re-colorise them as they see fit.

[averas]

@stevage Some great company is already doing so ... https://www.mapbox.com/blog/terrain-rgb/

[stevage]

Oh nice! :)

Another example of client-side recoloring code: TerriaJS

[cebartling]

@lucaswoj Do you have a timetable in place yet for providing this feature. We're trying to do some planning here at our company, and depending on when this feature may show up natively in Mapbox GL JS, we may need to resort to a plan B implementation of a product feature.

[lucaswoj]

@cebartling per our public roadmap, this feature is not slated for implementation in the next few months. Your options:

• We are always excited to help folks like you put together pull requests to add new features!
• The new canvas source type allows for integration with external raster manipulation. cc @lbud
• plan B

[lbud]

The new canvas source type allows for integration with external raster manipulation. cc @lbud

Right — the canvas copies image data directly from an HTML canvas onto the map (the same mechanism used for image and video sources), so the canvas can be colorized as desired. Here's an example that reads pixels from PNGs onto a canvas and attaches that to the map, colorizing as defined by the user (takes a few seconds to load all the images): https://bl.ocks.org/lbud/ee919cb9cf265c635e2adc899b65dbbb

I've played around with reading data from a tiled raster source already added to Mapbox GL (reading pixels from webgl tile textures, placing them into a subset of a fullscreen canvas, colorizing as desired and copying back to the map) but the amount of roundtripping combined with the complications of accessing raster tiles' webgl textures makes this approach more or less unusable. To do this "right" (tiled raster layers with color manipulation) would require either aforementioned raster-colorizer property or a custom layer type (https://github.com/mapbox/mapbox-gl-js/issues/281).

[cebartling]

Thanks for the information @lbud! I believe we're leaning towards a spike solution on the canvas source.

[sensoarltd]

Any updates on the release of this feature?

[SiggyF]

If I understand this issue correct, the implementation would consist of the following:

• [ ] add the raster-color style properties to src/style/style_layer/raster_style_layer_properties.js The variable raster-color would be consistent with for example the heatmap. Or is raster-colorize more appropriate? I'm assuming it would be of type DataConstantProperty<Color>.
• [ ] add the corresponding paint property raster-color to flow-typed/style-spec.js.
• [ ] add raster-color documentation and properties to the src/style-spec/reference/v8.json file, with "property-function": true, also zoom-function and transition?
• [ ] convert the colors to an ImageData/RGBAImage and then to a texture, similar to the implemenation of heatmap.
• [ ] add a uniform sampler2d with the color lookup to src/shaders/raster.fragment.glsl and lookup the color.

[andrewharvey]

@SiggyF That plan looks okay, but I'm not confident enough to comment. Would be very keen to see a PR implementing this!

The only thing I'd add is it'd be great to have it working on raster-dem sources to enable #6245.

[sk-]

Note that openlayers Raster source is more powerful as it allows to process the whole image at once, not only mapping pixels. This allows you to use the elevation data to compute the slope, which can be used to generate hill shading as in one of the linked examples or to render whole areas with a slope over certain threshold, useful for mountain sports.

[stevage]

Just discovered that you can do a reasonable workaround using Mapbox Street's "contours" layer as a fill, and interpolating color across elevations. Sounds crazy, but it actually looks ok!

Downsides include:

• doesn't work at low zooms
• works badly at medium-low zooms
• requires a bit of extra fakery to hide the gaps between contours. I added a thick line layer, with the same colouring, and a high blur level.

[shobhitg]

Interesting! Openlayers 3 has similar hooks to perform pixelwise operations on raster sources pre-rendering:

https://openlayers.org/en/latest/examples/raster.html https://openlayers.org/en/latest/examples/shaded-relief.html https://openlayers.org/en/latest/examples/region-growing.html

If implemented, this pixelwise operation is exactly what would help in #8080.

[shobhitg]

Is there any progress on raster-colorize?

If there is a mock implementation of raster-colorize, and if I can the same results as raster-stretch (#8587) then I am interested in utilizing that branch.

[andrewharvey]

Is there any progress on raster-colorize?

No, it's open for anyone who wants to implement it.

[shobhitg]

{ raster-opacity:1; raster-colorizer-default-mode: linear; raster-colorizer-default-color: transparent; raster-colorizer-stops: stop(0,#47443e) stop(50, #77654a) stop(100, rgb(85,107,50)) stop(200, rgb(187, 187, 120)) stop(255, rgb(217,222,170)); }

Just saying that like most settings in Mapbox, and in the spirit of OpenGL, stops should be ranging from 0 to 1 instead of 0 to 255. like...

{
  raster-opacity:1;
  raster-colorizer-default-mode: linear;
  raster-colorizer-default-color: transparent;
  raster-colorizer-stops:
    stop(0,#47443e)
    stop(0.2, #77654a)
    stop(0.4, rgb(85,107,50))
    stop(0.6, rgb(187, 187, 120))
    stop(1, rgb(217,222,170));
}

[andrewharvey]

Just saying that like most settings in Mapbox, and in the spirit of OpenGL, stops should be ranging from 0 to 1 instead of 0 to 255.

Those stops are dependent on your source data, so 8 bit bands would range 0-255, 16 bit 0-65,535 etc.

I'm in favour of this proposed feature working with stops from the source data, since you need to support palatted, ie. value 0 is red, value 1 is purple, value 2 is yellow etc. which wouldn't work if you rescale to 0-1.

[shobhitg]

Can someone clarify what raster-colorizer-default-color: transparent; means specifically in terms of color ramp (visual examples would be appreciated)?

Also it would be awesome if someone could try out in TileMill and post a sample grayscale terrain image file, and a color ramp info, and the exact expected outcome image. It would be good to have this info in order to test any possible solutions.

[andrewharvey]

Can someone clarify what raster-colorizer-default-color: transparent; means specifically in terms of color ramp (visual examples would be appreciated)?

If you're using a mode/interpolation value of exact, and your source data has a value which doesn't exactly match one of the steps, then it would use the default color to fill that.

I'd suggest if you are planning on implementing this, to put together a design document with the proposed style spec syntax first. You don't have to, but it would help.

[shobhitg]

Those stops are dependent on your source data, so 8 bit bands would range 0-255, 16 bit 0-65,535 etc.

I'm in favour of this proposed feature working with stops from the source data, since you need to support palatted, ie. value 0 is red, value 1 is purple, value 2 is yellow etc. which wouldn't work if you rescale to 0-1.

Other than supporting paletted colors this doesn't make much sense to me. A color ramp is easier to reason about when thinking between 0 to 1 instead of 0 to 255. Paletted colorization isn't something that needs to be supported. Does TileMill's color rasterizer support it?

If you're using a mode/interpolation value of exact, and your source data has a value which doesn't exactly match one of the steps, then it would use the default color to fill that.

raster-colorizer-default-color option feels unnecessary to me because transparent is default behavior, and in theory any other different color can be specified via color ramp using a stop value of 0 or 1 for leading or trailing edges of the color ramp.

[andrewharvey]

Other than supporting paletted colors this doesn't make much sense to me. A color ramp is easier to reason about when thinking between 0 to 1 instead of 0 to 255. Paletted colorization isn't something that needs to be supported. Does TileMill's color rasterizer support it?

You're right that it makes the most sense for paletted colors (which is common for landcover datasets: 0 is water, 1 is trees, 2 is sand etc), but even for other use cases it still makes sense to think about the input classes or steps to be in native raster values (not 0-1). For example working with a DEM, values are typically in meters above sea level, and its much easier to think about it that way than 0-1.

raster-colorizer-default-color option feels unnecessary to me because transparent is default behavior, and in theory any other different color can be specified via color ramp using a stop value of 0 or 1 for leading or trailing edges of the color ramp.

Yeah you could always just specify those other default values, but having a default value would make it simpler.

I'm not suggesting we copy raster-colorizer from CartoCSS just for the sake of it, I think it's fair to determine from scratch what exactly we want to support in GL JS and what that syntax would be.

I think the first step should be to try to document what the feature would look like and the syntax for the style spec, in terms of layers, sources, paint and layout properties.

[elfmanryan]

You're right that it makes the most sense for paletted colors (which is common for landcover datasets: 0 is water, 1 is trees, 2 is sand etc), but even for other use cases it still makes sense to think about the input classes or steps to be in native raster values (not 0-1). For example working with a DEM, values are typically in meters above sea level, and its much easier to think about it that way than 0-1.

I would second that using the real pixel values is important for a couple of reasons. Firstly, the two main GIS software packages (ArcGIS and QGIS + GDAL) scale using ramps and palettes built off pixel values. This is what many users are familiar with for a start, but also means that if someone wants to replicate a colour scheme it will be straightforward if the colour stops are also built of the inherent raster values in the same context; conversely, using 0-1 scaling will force a user to try and rescale their colour scheme to match! this will cause no ends of of frustration and compromise. Secondly, continuous data (e.g. float) unlike class/cardinal data (e.g. int) often represents real world values, such as tC/m2, or density of people etc. So when it comes to colouring, users will have natural stops and thresholds in mind that are real world specific.

With respect to stops; perhaps a good GIS precedent is gdal; te specific tool is gdal_dem (dont be mislead by the term dem, it colourises any raster continuous, classified outwith terrain); and uses a table of stops, but later releases allowed to-from rgb stops aswell. An example: https://gis.stackexchange.com/questions/308458/colorize-singleband-geotiff-raster-using-python-gdal-with-discrete-interpolation

[dazza-codes]

Nice to see a growing interest on the subject. Of course we (Ubilabs) would love to have this feature in mapbox-gl-js; we would still be maintaining our own fork (mainly for data-driven sdf icons), but this would lead to one thing less to worry about. I'm excited for your implementation! If you have questions about ours, we would love to help!

• @Scarysize - the obvious question is - why don't you (Ubisoft) submit a PR from your fork?