[FEATURE] OSM multiline relation handling

[msbarry]

Is there a general-purpose way to handle relations that define long connected lines (ie. route/waterway) in a similar way to how planetiler handles multipolygon relations today?

Polygons/multipolygons are currently handled transparently to profiles whether they came from a single way or a complex multipolygon relation. Relations are also used to group ways into long linestrings/multilinestrings, for example: hiking/biking/bus routes, waterways, country boundaries, public transit. Today, profiles need to process each way in the relation and emit a line, then merge connected lines when post-processing finished tiles. This doesn't let you process an entire connected linestring as a single geometry, for example to compute its overall length. It's also not supported by yaml schemas yet.

For example to process every national cycling route, a profile could do:

include_when:
  - network: ncn
geometry: linestring

Is something like that sufficient to cover most cases? If not how much more flexibility does it need? There are also super-relations (boundary and routes), cases where a way and relation it's contained in both match, and relation member roles that may convey importance.

The goal would be to make it possible to reconstruct multilines in a java profile (probably by extending multipolygon reconstruction logic) and expose a simple API for it through yaml configs.

[msbarry]

I'm not as familiar with the nuances of the different use-cases here, would appreciate feedback from others with more experience mapping these!

[zstadler]

The principles I'd like to preserve in the current semantics of the YAML configurations are:

1. An OSM element is selected if it complies with the include_when and exclude_when definitions. When applied to relations of any kind, it means that only the tags of the relation are taken into account.
2. The geometry is created by descending from the selected element down to the nodes level. In the case of linestring geometries of relations, the geometry is obtained by descending trough other super-/relations and ways down to the relevant nodes.
3. The tags are taken from the selected element

[msbarry]

Conceptually this makes sense. I'm wondering if we need additional flexibility to access and/or filter the ways that get included in the final shape by:

• tags on the ways
• role labels on the relation member elements
• tags on the relations contained in a super-relation (or superrelation member roles)

And also if there needs to be any built-in way to decide between a relation and superrelation, or if profiles can determine that entirely from the tags on them (for example see appalachian trail

[zstadler]

To accomplish that, in a general way, we could add the ability to refer to relation member role and tags. For example, extracting only some of the segments of a route

include_when:
  - network: ncn
geometry: linestring
include_members_when:
  - highway:
    - cycleway
    - path

or for extracting country labels

include_when:
  - admin_level: 2
geometry: point
include_members_when:
  - @role: label

Note: this is not necessarily the best syntax.

When such member-based valued are used as attributes, using some new syntax, the geometry would split the relation into separate MultiLineString features according to the different values.

As for relation members that are relations themselves, I'm inclined to treat them as transparent, and flatten any nested relations. That would make the ways and node members of any relation seem as if they are members of any predecessor super-relation.

[msbarry]

OK got it, then if a profile wanted to limit the relation type they would just put an extra type filter on include_when?

include_when:
  - admin_level: 2
  - type: boundary

I also wonder if we should make profiles opt-into this behavior? I'm trying to think if doing this by default would cause any issues if you're not expecting it?

[msbarry]

Also, we'd want to implement this capability in the java layer then have the yaml layer add an easier-to use API for it. I think it would make sense to build this as an extension to multipolygon handling, which currently does this:

pass1

• nodes: store the lat/lon of each node into nodeLocationDb disk-backed lookup table
• relations: store a set waysInMultipolygon that contains way IDs that are a member of a multipolygon relation

pass2

• ways: if a way is in waysInMultipolygon then store its geometry in multipolygonWayGeometries, a disk-backed map from way ID to node IDs
• relations: for multipolygon relations, get the list of node IDs for each way they contain from multipolygonWayGeometries then lookup the node locations in nodeLocationDb and reassemble the ways into a multipolygon

We want pass 2 relation processing to also be able to fetch node ids and locations for all of the ways it contains. The lookups should also be lazy so they only happen if profile decides the relation is relevant based on its tags.

For super relations we'd also need to be able to descend from the parent relation down to its children, which complicates things... For completeness we'd probably also want to expose super relation membership when processing ways.

[msbarry]

It we excluded super-relations, I think this could work by expanding waysInMultipolygon to be waysInRelation and either include all way IDs that exist inside a relation, or multipolygons + some filter provided by the profile. Then pass2 relation processing would have access to the geometries of each way it contains. For nodes contained in relations, we already have nodeLocationDb that can be used to look up their location. This just gives access to node and way geometries when processing a relation, but not their tags.

To include superrelations, we need to be able to descend to all children like this and get their geometries. I think this means we'd need to store an extra map of relation -> node/way/relation children since we can't depend on just having the relation info accessible because we're already processing it in pass 2 anymore. We could also limit the extra data we store to only relations contained in a superrelation if we did a third pass over relations, for example:

• relation pass 1: store set of relations that are a child of another relation
• relation pass 2: store relation info for every super-relation, and child relation
• relation pass 3 (after we finish reading OSM file): process all super-relations and their children stored during pass 2

I could try prototyping how much storage it would be for either of those options.

[msbarry]

For reference the current OSM planet pbf has:

• 12,042,918 relations
• 169,478 of them are superrelations
• they contain 18,781,152 nodes, 115,643,258 ways, and 863,384 relations
• there are 28,683 unique member roles

[Tristramg]

Thank you for examining this need.

I’m not very experienced with planetiler, so I’m trying only to give what I want to achieve, with no consideration how the schema would look like.

I want to build a map of the Parisian subway.

There are super-routes for each line: https://www.openstreetmap.org/relation/3328695#map=17/48.86859/2.31292&layers=T I need the colour tag for the rendering from the super-route, but I also need tags from the way (e.g. bridge).

So more than how to select the features, I would need some way to merge tags between the relation and the ways (and possibly also the role)

[caspg]

I have an use case where I need to have access to super relations. I'm building an app for cyclists where I'm presenting cycling routes.

Some of the routes in OSM are part of the "super relations". For example, we have EuroVelo 10 super relation which has other relations as members (and it can be nested).

• EuroVelo 10 top super relation OSM waymarkedtrails
• EuroVelo 10 part Poland super relation OSM
• EuroVelo 10 part Poland 3 OSM

planetiler only give me access to the final relation. Instead, I would like to have access to the parent relation and display only top super relation name.

Just wanted to share my use case.

[HarelM]

I've created the following PR:

• https://github.com/onthegomap/planetiler/pull/1341

Would be happy to get some feedback on it.

[zstadler]

For our reference, this could be the schema-equivalent of the Bike Route Overlay java example:

schema_name: Bike Paths Overlay
schema_description: An example overlay showing bicycle routes
attribution: <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a>
is_overlay: true
args:
  area:
    description: Geofabrik area to download
    default: monaco
  osm_url:
    description: OSM URL to download
    default: '${ args.area == "planet" ? "aws:latest" : ("geofabrik:" + args.area) }'
  osm_path:
    description: osm OSM input file path
    default: '${ "data/sources/" + args.area + ".osm.pbf" }'

sources:
  osm:
    type: osm
    local_path: '${ args.osm_path }'
    url: '${ args.osm_url }'

definitions:

  - bike_layer: &bike_layer
      merge_line_strings:
        min_length: 0.1
        tolerance: 0.1
        buffer: 4
      features:
        - source: osm
          geometry: line
          attributes:
            - key: name
            - key: ref

layers:

  - <<: *bike_layer
    id: mtb-route-international
    include_when:
      __all__:
        type: route
        route: mtb
        network: icn

  - <<: *bike_layer
    id: mtb-route-national
    include_when:
      __all__:
        type: route
        route: mtb
        network: ncn

  - <<: *bike_layer
    id: mtb-route-regional
    include_when:
      __all__:
        type: route
        route: mtb
        network: rcn

  - <<: *bike_layer
    id: mtb-route-local
    include_when:
      __all__:
        type: route
        route: mtb
        network: local

  - <<: *bike_layer
    id: mtb-route-other
    include_when:
      __all__:
        type: route
        route: mtb
        __not__:
          network: 
            - icn
            - ncn
            - rcn
            - local

  - <<: *bike_layer
    id: bicycle-route-international
    include_when:
      __all__:
        type: route
        route: bicycle
        network: icn

  - <<: *bike_layer
    id: bicycle-route-national
    include_when:
      __all__:
        type: route
        route: bicycle
        network: ncn

  - <<: *bike_layer
    id: bicycle-route-regional
    include_when:
      __all__:
        type: route
        route: bicycle
        network: rcn

  - <<: *bike_layer
    id: bicycle-route-local
    include_when:
      __all__:
        type: route
        route: bicycle
        network: local

  - <<: *bike_layer
    id: bicycle-route-other
    include_when:
      __all__:
        type: route
        route: bicycle
        __not__:
          network: 
            - icn
            - ncn
            - rcn
            - local