Source layer "park" does not exist on source "openmaptiles"
When running a build using a custom bbike export from extract.bbike.org, the park layer is missing, resulting in broken rendering of affected areas and the following errors in browser console:
Error: Source layer "park" does not exist on source "openmaptiles" as specified by style layer "park".
Error: Source layer "park" does not exist on source "openmaptiles" as specified by style layer "park_outline".
Reproduction
- Create custom area
- Put
builds/MyArea/.envin place with custom vars - Put
MyArea.osm.pbfin place
- Put
- Build docker images as in GHA workflow
- Build data assets
cd builds/MyArea && bin/build builds/MyArea- I see the planetiler container pull some endpoints of the format
https://onthegomap.github.io/planetiler-demo/#{z}/{lat}/{lon}/during this step
- I see the planetiler container pull some endpoints of the format
- Start local app
cd builds/MyArea && docker-compose up -f ../../docker-compose.yaml --env-file .env up -d- (Run
docker exec -it -u0 headway-pelias-elasticsearch-1 chown 1000:1000 /usr/share/elasticsearch/datato fix elasticsearch crashlooping on missing permissions)
- (Run
Env vars to use locally mirrored/forked sources for all assets are set both in builds/MyArea/.env and the local environment:
HEADWAY_PLANETILER_FIXTURES_URL=...
HEADWAY_IMAGE_REPO=...
HEADWAY_TILES_URL=...
HEADWAY_PBF_URL=...
HEADWAY_AREA=MyArea
planetiler log snip:
INF [archive] - Starting...
INF [archive] - Unable to load tile weights from data/tile_weights.tsv.gz, falling back to top 100k
[...]
INF [archive:write] - Shallow tiles written: 411
INF [archive:write] - Tile data written: 411 (0% omitted)
INF [archive:write] - Unique tile hashes: 0
[...]
last tile: ../..../... (z14 95%) https://onthegomap.github.io/planetiler-demo/#...
DEB [archive] - Tile stats:
DEB [archive] - Biggest tiles (gzipped)
[...]
DEB [archive] - Max tile sizes
#[list of water,boundary,aeroway,... but not park]
DEB [archive] - # tiles: 411
DEB [archive] - # features: 233,036
[...]
FINISHED!
----------------------------
data errors:
osm_multipolygon_missing_way 201
render_snap_fix_input 160
osm_boundary_missing_way 142
feature_polygon_osm_invalid_multipolygon_empty_after_fix 2
omt_park_area_osm_invalid_multipolygon_empty_after_fix 1
merge_snap_fix_input 1
Observations
- The issue is not present for the same area on https://maps.earth, where it renders with parks as expected
/tileserver/styles/basic/style.json- On
maps.earthand the local app,layerscontainspark. - This is controlled by
services/tileserver/styles/basic/style.json
- On
/tileserver/data/default.json- On
maps.earth,vector_layerscontainsparkandpark_outline - On local app,
vector_layersdoes not containparkorpark_outline. Otherwise the field looks identical to themaps.earthone. - It is not clear to me yet exactly at which point in the lifecycle
default.jsongets created (or if part of it's still somehow sourced externally)- Understanding the data-build lifecycle better would help. E.g. exactly what's the role of https://github.com/headwaymaps/headway-data and how were those files sourced/generated?
versionis3.15.0on local app;3.14.0on maps.earthplanetiler:versionis the same (0.7.0)- Using a newer planetiler version (
0.8.4) does not make a difference wrt the issue
- Using a newer planetiler version (
- On
Not clear if this is due to an application bug, misconfiguration, or broken data. I did redo the whole pipe on separate machines and exports with identical results.
Similar issue:
- https://github.com/mapbox/mapbox-gl-js/issues/10120
- Removing
parkandpark_outlinefromstyle.jsonas suggested by one comment removes the entries from the/tileserver/styles/basic/style.jsonresult but does seem not change behavior otherwise.
- Removing
The missing weights file was a red herring. If I change dagger/main.go to mount https://github.com/onthegomap/planetiler/blob/main/layerstats/top_osm_tiles.tsv.gz as data/tile_weights.tsv.gz for the Mbtiles stage, the error goes away but the end result is the same with park still missing.
Another error in the data-build logs, slightly further up for valhalla and possibly also unrelated:
Container.withMountedDirectory(
path: "/tiles"
source: Container.directory(path: "/tiles"): Directory!
): Container!
Container.withMountedDirectory DONE
Contaner.withExec(args: ["sh", "-c", "valhalla_export_edges -c valhalla.json > /tiles/polylines.0sv"]): Container!
Container.file(path: "/tiles/polylines.0sv") File!
Contaner.withExec(args: ["sh", "-c", "valhalla_export_edges -c valhalla.json > /tiles/polylines.0sv"]): Container!
[ERROR] (stat): /data/valhalla/tiles.tar No such file or directory
[WARN] Tile extract could not be loaded
[WARN] (stat): /data/valhalla/traffic.tar: No such file or directory
[WARN] Traffic tile extract could not be loaded
[INFO] Enumerating edges...
I don't immediately know the cause of this, but I can explain this part:
It is not clear to me yet exactly at which point in the lifecycle
/tileserver/data/default.jsongets created (or if part of it's still somehow sourced externally)
The tileserver is started with a config - see services/tileserver/configure_run.sh
The config contents includes a list of data sources. The default one for us is the planetiler generated mbtiles.
/tileserver/data/default.json is a dynamically generated response from the tileserver based on the contents of that mbtiles.
exactly what's the role of https://github.com/headwaymaps/headway-data and how were those files sourced/generated?
I've added a README just now: https://github.com/headwaymaps/headway-data
I don't think it's related to this issue.
Note how the parks layer (from style.json) is from the openmaptiles source, which, if you go to your data definitions at the top of the style file, comes from the "default" mbtiles (aka the planetiler generated vector set), not the terrain or landcover sources.
@michaelkirk Thanks for the pointers!
Note how the parks layer (from style.json) is from the openmaptiles source, which, if you go to your data definitions at the top of the style file, comes from the "default" mbtiles (aka the planetiler generated vector set), not the terrain or landcover sources.
I'm with you so far but where does the planetiler instance get those assets from? Looking inside https://data.maps.earth.example.com/planetiler_fixtures/sources.tar:
$ wcurl https://data.maps.earth.example.com/planetiler_fixtures/sources.tar
$ tar -tf sources.tar
lake_centerline.shp.zip
natural_earth_vector.sqlite.zip
water-polygons-split-3857.zip
I don't spot anything that seems related inside those zip files.
It seems like "park" in openmpatiles might have a much narrower meaning than what people would colloquially consider parks to be. https://github.com/openmaptiles/openmaptiles/blob/master/layers/park/mapping.yaml#L98-L103
Anything tagged leisure=park instead seems to be mapped to the "landcover" layer: https://github.com/openmaptiles/openmaptiles/blob/master/layers/landcover/mapping.yaml
@3nprob does your OSM extract have anything that would get mapped to "park", like a nature reserve or something? If not, something (planetiler possibly?) could be pruning the entire layer from the schema.
Thanks, I think you're on to it @ellenhp !
does your OSM extract have anything that would get mapped to "park", like a nature reserve or something? If not, something (planetiler possibly?) could be pruning the entire layer from the schema.
Looking with osmium show MyArea.osm.pbf, I see some leisure=park but no leisure=nature_reserve, boundary=national_park or boundary=protected_area.
Now to figure out how to use forked openmaptiles
IMO if you want to silence the error it makes the most sense to figure out how to prevent planetiler from pruning the layer, rather than making any schema changes.