Render tests

[wipfli]

Render tests can help with tracking changes across tiles and styles. We could make render tests for specific layers and areas. Let's make an example based on the Places layer.

Input OSM Data

First we need input data which does not change. If we were to use the latest data from OpenStreetMap, we would get rendering changes which are related to data changes and not to code changes in this repo. We could use tools like osmium tags-filter to extract some data from a given OpenStreetMap dump. We could do this on a thematic basis and also limit it spatially to keep file sizes small, say below some tens of megabytes. For the Places example we could look at low zoom levels z<=10. Here the relevant data would be captured by:

osmium tags-filter planet-latest.osm.pbf n/place=country,state,province,city,town -o places_lowzoom.osm.pbf

We need a way to store these osmium commands and distribute the output files. Maybe we could upload places_lowzoom.osm.pbf to R2, or we could commit it to this repo. The file size here is 11M.

PMTiles

Now that we have the input osm pbf, we can run planetiler on the places layer with something like:

wget https://my-r2-url.com/places_lowzoom.osm.pbf -O data/places_lowzoom.osm.pbf
mvn clean package
java -Xmx30g -jar target/protomaps-basemap-HEAD-with-deps.jar \
  --osm_path=data/places_lowzoom.osm.pbf \
  --output=data/places_lowzoom.pmtiles --force \
  --maxzoom 10 \
  --nodemap-type=sparsearray \
  --layer=places 

Running this command should only take a few seconds. Now we have a pmtiles file with just there relevant data for this places render test.

Actual Render Tests

We can now start running the actual render tests. We can follow the pattern of MapLibre GL JS, see https://github.com/maplibre/maplibre-gl-js/blob/main/test/integration/render/run_render_tests.ts

This uses pupeteer to render maps in a browser and create actual.png images for some locations. Those are compared to expected.png images using mapbox/pixelmatch, a library made to compare render test images.

We could have a list of locations where we want to render. For example in the Places layer, it could make sense to render Italy and Rome to check that the Vatican label does not collide away the Rome city label.

The expected.png images could be stored in this repository, or they could be stored in the same place on R2 as we could have the places_lowzoom.osm.pbf.

MapLibre GL JS has a convenient website to look at render tests locally. We could copy that one too.