Filling lakes, waterways, etc.

[ftheriault]

I am not sure if this is an issue with Tilemaker or if it is only from my understanding of the tool, but I seem to have issues filling river and lakes.

Perhaps if there was documentation on the proper config/process.lua to use when exporting water, the issue wouldn't occur. Anyone willing to share their proper config/process files to be able to correctly fill river and lakes?

The following image shows my rendering.

This image was rendered using OL3 and around the main area, it is supposed to be filled with water. Also, I have used Tilemaker against a GeoFabrik "North America/Canada/Quebec-latest.osm.pbf" file.

Any help would be greatly appreciated.

[TimSC]

Apparently, the second argument in way:Layer("layer_name", is_area) must be true (within process lua file). See CONFIGURATION.md for more info. I'm having difficulty getting areas to appear in my style... we might have the same problem.

[ftheriault]

I saw about the is_area parameter, and water was set to true... Perhaps we have the same issue.

I have tried many ways, but without luck. Here is a section of my current process.lua file. There must be extra data put in the tiles, but I had to try...

local waterway = way:Find("waterway") 

if waterway=="riverbank" or waterway=="river" or
    way:Find("water")=="lake" or
    way:Find("natural")=="coastline" or
    way:Find("natural")=="water"
then
    way:Layer("water", true)
    way:Attribute("name", way:Find("name"))
    way:Attribute("waterway", waterway)
    way:Attribute("water", way:Find("water"))
    way:Attribute("natural", way:Find("natural"))
end

[TimSC]

@ftheriault If you turn on verbose output, do you get anything?

[ftheriault]

Actually yes, I have so many lines similar to this :

Exception when writing output object 4294965929 of type 2

Any idea what it could be?

[systemed]

Could you share your current process.lua and config.json, so that we can try and reproduce this (with the Quebec extract)?

[TimSC]

I was just going to add I had a similar issue which I have since resolved - I'm just trying to remember the exact cause!

@ftheriault are you running the latest from the master branch? I'd be interested to see what happens if you try my https://github.com/TimSC/tilemaker/tree/otmdata branch as well.

[ftheriault]

Sure. This is my current configuration.

config.json

{
	"layers": {
                "landuse" : { "minzoom" : 8, "maxzoom": 15},
                "water": { "minzoom": 8, "maxzoom": 15},
                "roads": { "minzoom": 10, "maxzoom": 15, "simplify_below":13, "simplify_level":0.001, "simplify_ratio":2.0 },
                "buildings": { "minzoom": 13, "maxzoom": 15 },
                "pois": { "minzoom": 13, "maxzoom": 15 }
	},
	"settings": {
		"minzoom": 10,
		"maxzoom": 14,
		"basezoom": 14,
		"include_ids": false,
		"name": "Montreal",
		"version": "0.1",
		"description": "Montreal Vector Tiles",
		"compress": "none",
		"metadata": {
			"json": { "vector_layers": [] }
		}
	}
}

process.lua

-- Nodes will only be processed if one of these keys is present

node_keys = { "amenity", "shop" }

-- Initialize Lua logic

function init_function()
end

-- Finalize Lua logic()
function exit_function()
end

-- Assign nodes to a layer, and set attributes, based on OSM tags

function node_function(node)
	local amenity = node:Find("amenity")
	local shop = node:Find("shop")
	if amenity~="" or shop~="" then
		node:Layer("pois", false)
		if amenity~="" then node:Attribute("type",amenity)
		else node:Attribute("type",shop) end
		node:Attribute("name", node:Find("name"))
	end
end

-- Similarly for ways

function way_function(way)
	local highway = way:Find("highway")
	local building = way:Find("building")
	local landuse = way:Find("landuse")
	local waterway = way:Find("waterway")

	if highway~="" then
		way:Layer("roads", false)
		way:Attribute("name", way:Find("name"))
		way:Attribute("type",highway)
--		way:Attribute("id",way:Id())
--		way:AttributeNumeric("area",37)
	end

	if waterway=="riverbank" or waterway=="river" or 
		way:Find("water")=="lake" or
	 	way:Find("natural")=="coastline" or
	 	way:Find("natural")=="water"
	then		
		way:Layer("water", true)
		way:Attribute("name", way:Find("name"))
		way:Attribute("waterway", waterway)
		way:Attribute("water", way:Find("water"))
		way:Attribute("natural", way:Find("natural"))
	end

	if building~="" then
		way:Layer("buildings", true)
		way:Attribute("name", way:Find("name"))
		way:Attribute("type", building)
	end

	if landuse~="" then
		way:Layer("landuse", true)
		way:Attribute("name", way:Find("name"))
		way:Attribute("type", landuse)
	end
end

I have tried two different extracts, with the same behavior. They can be found here:

• https://s3.amazonaws.com/metro-extracts.mapzen.com/montreal_canada.osm.pbf
• http://download.geofabrik.de/north-america/canada/quebec-latest.osm.pbf

Thanks for the help!

[TimSC]

Looks like there are bad geometries in the water polygons (running on 2cbb4a5bfc5eaf46983f6a7f922cfc1d0ac7282c):

Layer landuse (z8-15)
Layer water (z8-15)
Layer roads (z10-15)
Layer buildings (z13-15)
Layer pois (z13-15)
Reading montreal_canada.osm.pbf
Block 7 group 0 ways 0 relations 10780         
Stored 3771759 nodes, 148722 ways, 292 relations
Error in buildWayGeometry: Geometry has invalid self-intersections. A self-intersection point was found at (-74.5519, 51.9306); method: i; operations: i/u; segment IDs {source, multi, ring, segment}: {0, 0, -1, 0}/{0, 0, -1, 262}
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_get> >'
  what():  boost::bad_get: failed value get using boost::get
Aborted

Apparently the dissolve algorithm has been implemented in boost which can fix polygons. It seems to be an "extension" to boost; I can't find it on my system...

[systemed]

Yep, at https://github.com/boostorg/geometry/blob/develop/include/boost/geometry/extensions/algorithms/dissolve.hpp . It might be worth trying to work this in.

[ftheriault]

I am happy to know you guys were able to understand the issue! The thing is... what should I do now? Recompile boost on my system with this extension?

[TimSC]

@ftheriault Your data is tougher than what I've tried so far. I've added additional exception handling to 7b1cf31eb33c8361a5c5e26a9ce0a670f7f608fb so any problematic data is ignored. We still need to get to the bottom of what is going on with self intersections in polygons.

I also modified your style so linear features remain linear:

	if waterway=="riverbank" or
		way:Find("water")=="lake" or
	 	way:Find("natural")=="water"
	then
		way:Layer("water", true)
		way:Attribute("name", way:Find("name"))
		way:Attribute("waterway", waterway)
		way:Attribute("water", way:Find("water"))
		way:Attribute("natural", way:Find("natural"))
	end

	if waterway=="river" or 
	 	way:Find("natural")=="coastline"
	then		
		way:Layer("water", false)
		way:Attribute("name", way:Find("name"))
		way:Attribute("waterway", waterway)
		way:Attribute("natural", way:Find("natural"))
	end

[ftheriault]

Alright. Also, thanks for the process.lua fix.

[TimSC]

I had trouble getting boost's dissolve to compile. It is not part of a released boost but has been in the develop branch for a few years. I looked around for alternatives and noticed mapbox uses the clipper library https://github.com/mapbox/mapnik-vector-tile/issues/53 . I have used it to try to repair self interesting polygons (fd6be62) and this seems to work.

There seems to be further errors when processing the Canada data, possibly related to unexpected unclosed ways. I'll continue looking...

Update: Looks like a minority of self intersecting polygons are being fixed at the moment.

Update2: Ok, boost is_valid seems to be unreliable. As far as I can tell, the polygons really are fixed...

[systemed]

@TimSC Now that we've got support for riverbank-style multipolygons, I've been encountering a few edge-cases with Clipper. Sometimes the 'hole' disappears from the multipolygon: I've encountered this for the islands in the River Cherwell in Oxford, and the River Coln near Bibury, for example (using OSM Oxfordshire and Gloucestershire extracts). I think the issue is possibly with ConvertToClipper and ConvertFromClipper rather than Clipper itself, but for the life of me I can't see anything wrong with either!

However, using Boost::Geometry's intersection (as we already do for MultiLinestrings) actually seems to work fine in these circumstances. Any thoughts? I'm guessing there was a good reason you chose Clipper in the first place and I just haven't twigged it yet!

[TimSC]

@systemed I introduced clipper because it seemed to produce a more reliable result (ironically). I'll take a look at this again later today...

[systemed]

Heh - nothing's ever simple...

For a quick demonstration: https://map.canal.travel, you'll see the blue lake to the right of the built-up area in Oxford. At z12 it's fine (it doesn't cross tile boundaries so we don't need to clip it), but at z13/14 the island disappears.

[TimSC]

I ripped out the clipper code as a test but the problem still seems to occur. My branch: https://github.com/TimSC/tilemaker/tree/noclipper I'm not sure what the problem might be now!

(btw, the code runs 100x faster without combining similar objects/clipping at the edge of tiles, which is what clipper is used for. Perhaps we want to see if we can optimize this code further.) Update: already noticed here #115