Feature Properties Transform

[wipfli]

The things that a user can do with MapLibre style expressions based on feature properties is limited to the implemented styling language. While the style language has support for basic math, logic, and rudimentary string operations, many things cannot be done easily.

For example, it is hard to format a number to the locale India with 3 significant digits, e.g.: turn 1434389.043 to 14,30,000.

Another example: it is not possible to extract a value from a JSON encoded dictionary.

Both of these tasks are trivial to achieve in JavaScript or any other programming language and we usually motivate users to do such transformation at tile generation time.

In the past, we rejected a proposal to reference user-defined javascript functions from the style.json.

What I would like to introduce instead is a user-defined Feature Properties Transform, a callback which gets executed during vector tile parsing.

The Feature Properties Transform gets executed on every feature and has access to the following data:

• source name
• source layer name
• tileID z/x/y
• feature properties

With this information, one can easily filter based on source and source-layer, like people do usually in style.json documents and since the transform has access to all feature properties, it can do arbitrary logic and execute javascript on the properties dictionary. This is very powerful and opens the way to an alternative way of styling maps, namely more styling in JavaScript and less in the styling language.

Proof-of-concept implementation: #4199

[HarelM]

I think this is a very good idea for a hook. We need to remember that allowing things to be only in javascript will cause issues with parity between native and web. This is true for add protocol as well, but addProtocol forces you to put something "Wrong" in the style as opposed to this feature which you might just get an empty label and this can "fail" siliently.

In any case, these hooks are important and people using them should know that they create issues with parity.

One thing to keep in mind looking at the PR is that the transform is done in the worker thread while the map API is available in the main thread, and passing a method to the worker is very hard (you can probably pass very basic method, without the context of the main thread). So there needs to be a way to pass a method to the worker, which is only using importScript, which needs a javascript file or a string converted to dataURI.

In any case, I think adding that hook, since it is in the worker can be done differently maybe, using some "noop" method you override in the worker tile class - i.e something like

class WorkerTile {
...
    transfromFeature(feature) = return feature // this is a new noop method

}

and call this code in the worker context:

WorkerTile.prototype.transfromFeature = (f) => {//do something else with f and return it }

IDK, worth prototyping to see what's the easiest approach.

[wipfli]

Thanks for the feedback.

Here is an example how to use the proof-of-concept: https://github.com/wipfli/maplibre-feature-properties-transform-example

And you are right @HarelM the setFeaturePropertiesTransform function should not be available in the main thread, only in the worker...

[HarelM]

This looks elegant. Nice work!

[neodescis]

This would be great! I just went to a lot of work to implement color hashing of features based on some feature value. Without this, I had to get a list of unique values up-front from another endpoint, hash those, and put them all into a (sometimes giant) case expression. With this addition, it would be just a few lines of code.

[neodescis]

@wipfli , is this still something you're wanting to do? I have another need for something like this, and this seems better than going the addProtocol route.

[wipfli]

Thanks for asking @neodescis! I think this would be a good addition but I did not have a use-case for it recently and so did not work more on it. We wanted to use queryRenderedFeatures for testing but then found out that the output of that function is not consistent with the feature properties transform. So before moving on we should understand better how queryRenderedFeatures works and why the feature properties transform is not respected. Also we will need a way to test it... Maybe the solution is as simple as documenting the limitations of queryRenderedFeatures. I think it does not respect symbol-sort-key ordering on symbol layers, which it does respect it for circle or line layers and I am not sure if such things are documented. Would you be interested in pushing the pull request forward @neodescis ?

[neodescis]

Thanks for the additional information @wipfli! I am indeed interested in pushing the pull request forward. I may not get to it for a few days, but this has become somewhat of a priority for what I am working on. I have some unfortunate life events I am working through at the moment, but hopefully I can start digging into the pull request soon.

[neodescis]

@wipfli , I've had the time to go through your pull request. Is there a reason this approach would be limited to just the features' properties, and not their geometries as well? I realize it is working within the constraint of a tile, but I have another use case that involves transforming a point with a radius in properties into a "circle" (as approximated by turf). I'm currently doing this server-side when generating the tiles, but this doesn't scale well for thousands of features, as the "circles" become large in terms of data transfer.

The constraint of having to do this via importScripts() is a bit annoying in terms of API, but not impossible to overcome. I do imagine it will generate questions from users though.

At any rate, I'll play with it and see what I can come up with.

[wipfli]

You should look into geometries. That is a great idea. I just did not have a use-case for this at the time...

[HarelM]

Now that reading the geometries has its performance considerations since geometries are lazy loaded.

[neodescis]

@wipfli, I'm guessing you were running things in Firefox, is that correct? It seems the example doesn't load in Chrome (v131), at least for me. I'm seeing this:

blob:null/40caae4e-deac-4ed5-ba6c-d97efc5b3db9:37687 Not allowed to load local resource: blob:null/3ae48369-c942-4618-a2fe-26eb4a692cba
Uncaught (in promise) NetworkError: Failed to execute 'importScripts' on 'WorkerGlobalScope': The script at 'blob:null/3ae48369-c942-4618-a2fe-26eb4a692cba' failed to load.

It works for me in Firefox though. It must not have the same restriction on loading a blob URL from a file:/// origin.

Edit: It seems that a data URL works fine in both browsers though, so probably I'll switch at least the example to use that.

[wipfli]

I use chrome on ubuntu. It is an old version I am ashamed of but somehow automatic updates are not working on my laptop... Feel free to open an issue in https://github.com/wipfli/maplibre-feature-properties-transform-example?tab=readme-ov-file if one of the examples is broken...

[neodescis]

I've actually already pulled your branch into my own fork, and I've started working on it there. I was going to create my own PR if I get somewhere with this. Hope that's alright.

I've debugged into queryRenderedFeatures and discovered that it is ultimately creating new VectorTileFeature instances on the main thread, re-parsing the protobuf data for each feature as part of the queryRenderedFeatures call. I'm going to look into whether it's possible to use the features returned from the worker instead, but I'm not sure if that's possible yet.

[neodescis]

Closed by https://github.com/maplibre/maplibre-gl-js/pull/5370