openstreetmap
Migrate new vector layers from maplibre-gl-leaflet to “unwrapped” maplibre-gl-js
Problem
It’s really exciting to have vector tiles hosted for the first time on openstreetmap.org!
Currently, vector tiles are still displayed using maplibre-gl-leaflet, which wraps maplibre-gl-js in the same Leaflet API that we use to display raster maps like Carto.
Using the leaflet wrapper for the vector maps holds the vector maps back in a container that’s designed around the constraints/limitations of raster maps:
- No “smooth zooming” between integer zoom levels
- No ability to change map bearing (rotation) / pitch
Description
It would be great to migrate from Leaflet to mapbox-gl-js. There are several pieces that would need to be migrated to different APIs, some of which are documented by @tordans in https://github.com/gravitystorm/openstreetmap-website/issues/289.
Theoretically, using maplibre-gl-js directly we could:
- support smooth zooming, a big user experience improvement that will help the map feel more modern and responsive.
- support rotating the map
- support potential future features, including but not limited to:
- clickable POIs and features that can surface simple data without a round-trip to overpass.
- a 3D globe projection at low zoom levels
- Interactive map features that are inside the map instead of just on top, for example a routing line that appears faded when it passes under a tunnel or through a building
- extruded 3D buildings based on building height
- the ones listed here by @1ec5
- “modernize” the front-end a bit by using a more contemporary map renderer / javascript library
As I explained in the other ticket while we'd love to do this I strongly suspect that maplibre-gl-js is missing many of the features we need at the moment. I did a test re-implemenation of my personal site at maps.compton.nu a while ago and found it very hard to get even close to feature parity.
maplibre-gl-leaflet is a wrapper around maplibre-gl-js, so to my understanding any features it implements are supported by maplibre-gl-js.
do you have any examples of the missing features we need?
A not necessarily complete list of things we need:
- Ability to overlay markers with optional popups, including draggable markers for note addition and route endpoints
- Ability to overlay arbitrary geometries for routes and data visualisation
- Ability to overlay a draggable and resizable box for export area selection
- Interface to browser geolocation (though I guess we could do this ourselves)
We also have a load of stuff like most of the right hand side panels which although they're mostly custom are built on top of the leaflet framework as leaflet controls although they probably don't really need to be.
It was 18 months ago when I worked on this for my site, and I do use some other things, but I really found it very painful working with maplibre compared to leaflet.
I’m optimistic that all of these things can be built efficiently with maplibre-gl-js, and many of them can even provide better interoperability with the map in a “vector-native” environment compared to with raster tiles!
- Draggable markers, markers with popups
- Overlaying arbitrary geometries
- side note: one advantage in a client-side rendered vector map stack is that these types of overlays can fit “inside” the map instead of simply appearing on top—for example a route line could be drawn with a different color treatment where it passes under a “buildings” layer
- Overlaying a draggable and resizable box
- Interface to browser geolocation
As an aside, although I haven’t contributed to this codebase in the past, I have some experience building interactive vector map applications with these tools, and I’d be enthusiastic about volunteering time + effort to this project in a pull request.
How to use the just-introduced vector-based marker definitions with MapLibre is also a point I'm unsure about, considering the CSS- and DOM-driven coloring.
Happy for you to give it a go - it's going to be a major project and a pig of a PR because there's no realy way to break it down into small steps so it will have to be one massive PR that changes everything in one go 😢
The port to Leaflet in #121 had nearly 2.4k changed lines (+ 9.4k of leaflet into vendor), and I think this port will be significantly bigger...
tbf most of that diff was the addition of leaflet itself to vendor, but agree it’s a big effort.
smooth zooming, a big user experience improvement that will help the map feel more modern and responsive.
Leaflet has a zoomSnap option, fractional zooms, and Leaflet.SmoothWheelZoom would address the vector ux while still allowing 1:1 pixel representation for raster maps that seems IMO more difficult to get right otherwise.
supporting higher zoom levels
If you need zoom levels higher than 23, I think you might be looking at the wrong project.
The current title of this issue seems to suggest the the new vector layers can be migrated to MapLibre GL JS independently while keeping Leaflet around for the raster layers. I imagine using two entirely separate map display libraries on the same page would be a bit of a nightmare and that is why the currently solution using the Leaflet Maplibre plugin was chosen. I expect the sane path off of the plugin would be to replace Leaflet entirely with MapLibre (it also supports raster layers). When phrased like this the project is clearly much larger as it would require replacing everything below the header on the home page, not just the vector layer display code.
My original idea was to “use two separate map display libraries on the same page” — we could switch out / instantiate the underlying map implementation when the user switches map layers, and we could adjust front-end code that interacts with the map instance so that it had implementations for both map renderers.
However, looking at the code, that could easily turn out to be silly and/or infeasible.
maplibre-gl-js is a much more “modern” javascript library than Leaflet is, and has the potential to be much more performant at rendering, and provide a richer feature set down the road. So, I do believe that moving to maplibre-gl-js for all layers would be an improvement; however, that would be contingent on an update to @pnorman’s analysis of whether a sufficient proportion of users’ browsers have webgl support.
I absolutely don't want to try and run both and swap between them - aside from anything else that means having two implementations of everything else - two ways to overlay a route, two ways to overlay OSM data, two ways to draw markers for notes, etc, etc.
I think you’re right; I’ve updated the original issue, and added additional details
Thanks for opening this ticket and starting the conversation about a full migration to GL JS. OpenHistoricalMap’s fork of this project has been using maplibre-gl-leaflet for some time to avoid deviating too much from OSM, but this plugin’s limitations become apparent pretty quickly.
I agree that we should target a single map library if possible. In theory, a dual-library setup would enable us to keep supporting non-WebGL-capable environments, but I’d expect the code maintenance overhead to be substantial. We could make our own mapstraction library on top of both, but that would get us right back to square one with maplibre-gl-leaflet.
How to use the just-introduced vector-based marker definitions with MapLibre is also a point I'm unsure about, considering the CSS- and DOM-driven coloring.
There are two ways to put a “pin” on a MapLibre map:
- A DOM element using the marker API. This would be pretty similar to what we’re already doing.
- A symbol layer using the runtime styling API. This is the traditional MapLibre approach with better performance and the ability to have the symbol underlap other layers, but interactivity is harder to implement. This is the same API we’d use to overlay an element’s geometry or a route line on the map, but it would probably be overkill for just a marker.
keep supporting non-WebGL-capable environments
I think this argument fails at the layers pane, see #6240.
Well, in principle it’s possible to feature-detect webgl support, and then skip initializing the client-rendered maps when the layers pane is opened where webgl is unsupported.
99% of users visiting the front page in July had WebGL support. This is a similar number to ES6 support. I don't know that non-WebGL-capable environments need to be a priority.
Aside from all the positive benefits of migrating to MapLibre for real, maplibre-gl-leaflet has some serious drawbacks that reflect poorly on the website. For example, the jittering and tearing in openstreetmap/openstreetmap-website#6226 openstreetmap/openstreetmap-website#6233 is specific to this plugin. While it has gotten worse with recent versions of MapLibre, similar issues have been raised against the forked plugin and the original unmaintained Mapbox project for several years: mapbox/mapbox-gl-leaflet#143 maplibre/maplibre-gl-leaflet#26 maplibre/maplibre-gl-leaflet#63.
More worryingly, there’s an elusive bug where some tiles don’t draw until something forces a redraw of the tile, or never redraw at all. In OpenHistoricalMap, we’ve been seeing unreproducible reports of it for years, and now similar reports are coming in for this site too. I suspect it’s a race condition stemming from how the plugin works, but I haven’t looked into it further. It seems to happen more frequently when loading large tiles or doing more with runtime styling. Users will naturally attribute the lag to our tile servers being unresponsive rather than this rendering bug.
99% of users visiting the front page in July had WebGL support. This is a similar number to ES6 support. I don't know that non-WebGL-capable environments need to be a priority.
Note that MapLibre GL JS is very likely going to switch to WebGL 2 in the (near) future.
Here are the engineering reasons behind why "we" are considering WebGL2 / WebGPU:
- https://github.com/maplibre/maplibre-gl-js/discussions/6017
The adoption numbers from Microsoft don't look great on that feature 😓.
Not a maintainer on that project, but I suspect that we will discuss this on our TSC meeting => https://maplibre.org/community
Unfortunately matomo doesn't give us any numbers for WebGL versions, just WebGL as a whole.
Not a maintainer on that project, but I suspect that we will discuss this on our TSC meeting
@CommanderStorm any idea if WebGL 1 deprecation was discussed at all in the last TSC meeting?
We can probably get webgl 1 vs 2 by browser numbers. With webgl 1 the concern was browsers that could in theory support it but it was turned off or broken on the user's computer
It may be more complicated than that though... I know firefox blacklists various GL things sometimes depending on things like what graphics card you have.
Here is the discussion on the MapLibre side: https://github.com/maplibre/maplibre-gl-js/discussions/6017
If WebGL 1 is a requirement for OpenStreetMap to adopt MapLibre you could just stick with version that supports it. You see people use 3-4 year old versions of the library in the wild. It's not going to break randomly, you just don't have the latest features and performance optimizations.
I am pretty confident that if a big project like OpenStreetMap requires WebGL 1 in the foreseeable future, there will be other projects, and there may be some capacity to facilitate back-porting and putting out releases for an older version.
I did a bit of tinkering about half a month back. The main problem that I stumbled upon is that keeping this reviewable is kind of a struggle. Doing this in one shot will shurely not be acceptable due to PR size.
Doing the necessary changes I noticed how many tiny corners there are and that replacing this would be sure to break at least one of them. -> To keep risk Manageable for first OHM, second OSM some feature flag system (be it compile or runtime) would likely be needed to keep risk down and "the build green".
That is something where my ruby knowlege failed me. Is there a PR I can look at for feature flags?
I think such a migration would defintively need to be behind a feature flag, given that it changes most of the map related code (for example because leaflet and maplibre do lat/lng vs lng/lat, subtle to miss 😆).
I am not super confident that I can make that part work tbh, ruby is not taught at uni and it is kind of weird. Any pointers (or pair-programming) appreciated.
Also I had to hack parts of the ruby environment (f.ex. config.assets.debug in config/environments/development.rb) together to get the frontend symbols to be unmangled, not sure if there is a guide how to do this properly or if this is a.
@CommanderStorm - I may be able to help. I am experienced with Rails, although the asset management has always been a tricky part of it.
At the moment there are no feature flags, but something could be added if we think it's worthwhile. There are many options, we can look into what's feasible.
Do you have a branch where I can have a first look? Then perhaps we can do some pairing.
I'm not sure I made entirely clear but the intention of my initial comment here was to make clear that while we're interested in migrating to native maplibre we would want to do it for all layers - we don't want to be trying to switch between leaflet for raster layers and maplibre for vector layers.
A follow on from that is that we wouldn't expect to have leaflet present anymore afterwards let alone keeping both operable with a configuration option to switch between them.
I agree that PR size is likely to be an issue but I think it's going to be hard to have any way to make a change like this in an incremental fashion.
I'm not familiar with config.assets.debug but in development mode you should (if you're doing things right) normally get the original javascript without any optimisation.