Which symbolizers have support for render-time variables? Unclear from the docs

[davenquinn]

trafficstars

I don't see anywhere in the node-mapnik or mapnik docs for the supported symbolizers for render-time variables. Your writings make it seem as if all symbolizers are supported as of Mapnik 3, but it appears that the raster-colorizer stops cannot be used with render time variables (for instance).

This might be in error and due to something in my code, but I am not sure (because there is no reference to where they are supported).

For this case, variables work fine in the line-width field but not in the <stop /> tag (neither the value nor color fields. I am running mapnik 3.0.13

[springmeyer]

The properties with "expression":true in the mapnik-reference document is how to know which properties support expressions. Look inside https://github.com/mapnik/mapnik-reference/blob/gh-pages/3.0.6/reference.json. That doc is used to generate the more friendly docs at http://mapnik.org/mapnik-reference/#3.0.6, but it looks like that is missing the display of this expression:true property.

Pull requests welcome that modify https://github.com/mapnik/mapnik-reference/blob/gh-pages/site/main.js to display this property in the docs.

[davenquinn]

Hi, that's a good tip. I may submit a request for that soon.

Now that I've scrolled through the file, I see that expressions do not work for any of the raster symbolizers. Is there a major technical reason for this?

Having variable support here would be huge for my work with Mars elevation data, as I could actively tune the bounds of elevation colorization based on min-maxes per image (or responsively darken/lighten image data). I can modify the mapfile to entering mapnik if needed, but layering on another variable interpolation step seems overcomplex.

If there were no major barriers to this functionality, I'd be happy to make an attempt at the code.

[springmeyer]

Now that I've scrolled through the file, I see that expressions do not work for any of the raster symbolizers.

Correct.

Is there a major technical reason for this?

Yes. Expressions are evaluated, at the feature level, against mapnik::values (a recursive variant type). A mapnik::value can come from 3 places:

• Feature properties (from the data being pulled from a given mapnik plugin)
• Geometry types (from the features)
• Variables passed into the renderer at runtime

The way that Mapnik handles rasters is my attaching the pixels to a given feature. A layer in Mapnik with a raster always has one feature and no more. This feature does not have a geometry. And rasters also don't have properties in the same sense as vector data. Currently the only "properties" on a raster is a boolean of whether the raster has a nodata value. See the feature->put at https://github.com/mapnik/mapnik/blob/master/plugins/input/gdal/gdal_featureset.cpp#L595.

So, in short because rasters don't have meaningful properties at the feature level (whole image level) we've not attempted to make these available for styling. But theoretically it would be possible to make expressions work. The limitation is that expressions would allow for filtering rasters but not colorizing them - again because expressions work per feature (and mapnik stores a raster in a feature) and not per pixel currently.

[springmeyer]

Having variable support here would be huge for my work with Mars elevation data, as I could actively tune the bounds of elevation colorization based on min-maxes per image (or responsively darken/lighten image data).

So, if I an understanding right, you want:

• 1. Ability to evaluate those variables in raster colorizer
• 2. Ability to have variables available that represent summary statistics on the whole raster

Is that correct? 1. would be involved but probably feasible by making all properties of the raster colorizer expressions so they were evaluated at runtime (https://github.com/mapnik/mapnik/blob/master/include/mapnik/raster_colorizer.hpp). 2. would be more tricky to design correctly because:

• calculating statistics could be expensive (would need to be done when gdal_datasource is created)
• It would be hard/impossible to know which statistics to pre-calculate

So, an alternative design would be to calculate the statistics yourself, outside of mapnik, and then pass them down through the renderer. Variables passed to the renderer are passed to the mapnik::query object. Then mapnik::query.variables() could be accessed in the gdal_datasource similiar to how it is done in the postgis datasource: https://github.com/mapnik/mapnik/commit/e300a41dca946ddc588339554b7fe376c112101c. The limitation here is that variables passed into the renderer are per render and apply to all layers. So the same value would be passed to all raster layers in your stylesheet. To the extent that you want to render multiple layers at the same time that might have different stats this would not work.

[davenquinn]

Thanks, Dane. With this I was referring exclusively to your point 1., due to the issues you bring up in the second half of your reply. I know of no GIS system which does 2. in a flexible manner. I agree that calculating statistics outside of mapnik is the way to go for this (easy in PostGIS or precalculated with, e.g. rasterio over the window of your choice).

My envisioned "simplest" case is shown below: several elevation maps in different height bands. I know the min/max for each scene and would like to apply these with raster-colorizer stops, without writing a different XML config for each image.

On a related note, one potential difficulty -- is there a way to set default variables at the datasource level? I don't think that's a part of the XML spec but it would be nice to have.

[springmeyer]

On a related note, one potential difficulty -- is there a way to set default variables at the datasource level?

No, but maybe I'm not following. Can you say more? Do you mean, by providing default variables as datasource options? Or in the Stylesheet?

[davenquinn]

Either/or: I know in a carto stylesheet you can set default variables, but these are evaluated during the compilation of the stylesheet. I'm talking about defaults that would be evaluated at runtime, to set values of expressions that are not provided to the render() method programmatically.

Basically, when you author a stylesheet with expressions, you have to be sure that expression would be provided by the datasource. This inhibits the flexibility of say, having a raster-symbolizer with default stops of 0 and 255 (basically an identity transform for many bit-rasters) and optionally passing new stops if you want to darken/lighten the data.

It's not too bad to always pass in default variables, but specifying them with the rest of the style would be nice. They could be implemented as a Defaults child of the Style element? Anyway, this seems like a major addition if not already provided for in some way.

[springmeyer]

can set default variables, but these are evaluated during the compilation of the stylesheet.

Yes

Basically, when you author a stylesheet with expressions, you have to be sure that expression would be provided by the datasource.

Yes, against. You've got it, and thanks for explaining further so I could follow exactly.

They could be implemented as a Defaults child of the Style element? Anyway, this seems like a major addition if not already provided for in some way.

Yes, this would be tricky to design right. Currently we don't have support for defaults, and when an expression fails due to a missing property we fallback in the most reasonable way possible given the code at hand - and yes, this might not be optimal for a given usecase.

See my mention of this at https://github.com/mapnik/mapnik/issues/2553

[davenquinn]

OK, sounds good, reasonable fallbacks are a good solution for the time being. If a clear benefit comes up this might be worth revisiting.

As for the raster-colorizer-stop expressions, I'd be excited to see an implementation of this. I'm not a good C++ developer but if there's a clear reference case I can use for what the runtime-evaluated expressions would look like, I could experiment. Would there be big performance implications? Any advice would be appreciated, or I might just poke around.

Also, I submitted a PR to the docs to incorporate expression support as a badge.

[springmeyer]

@davenquinn - checking in here. Any chance to work on an implementation? Anything else left to do here before closing?