brouter-web icon indicating copy to clipboard operation
brouter-web copied to clipboard

Published 20 hours ago • verified publisher icon nrenner

OpenCycleMap and Outdoors alternatives?

Open nrenner opened this issue 6 years ago • 11 comments

I got a notice from Thunderforest that our tile usage (about 1,340,000 tile requests per month) is exceeding the free quota and that we need to upgrade to the "Solo Developer" plan, which is £95 (107 €) per month.

I'm aware that running a tile server is demanding and expensive, but at the moment I'm neither willing nor able to pay such an amount, especially as BRouter-Web is non-commercial and not making any money.

So I'm looking for alternative tile services with world-wide coverage and current data to add to the layer switcher, that can divert the traffic or replace the OpenCycleMap and Outdoors layers.

The idea is to also provide a separate list, where the user can choose additional layers from. These can also be regional/specific maps that are only of interest for part of the users. Below a list of candidates I found.

@ all: Do you know some more?

Commercial providers

Out of interest I did a a pricing comparison of some commercial tile services. None of them seems to include bicycle/hiking routes in their outdoor styles.

Provider Free tiles¹ Our price¹² Pricing¹² Outdoor styles
Thunderforest 150,000 107 € 1,500,000 tiles = £95 OpenCycleMap, Outdoors
Mapbox 750,000³ 18 € ⁵ $0.50 per 1,000 map views⁴ > 50,000 = 20 $
(1340000/15-50000)/1000*0.50		 Outdoors
MapTiler Cloud 100,000 55 € 20 $ + $0.05 per 1000 tiles > 500,000 = 62 $
20+((1340000-500000)/1000*0.05)		 Topo
GEOFABRIK - 140 € 1,000,000 - 10,000,000 tiles Topo
Omniscale - 110 € 750,000 - 2,000,000 tiles Outdoor
Jawgmaps 750,000³ 50 € ⁵ 1,500,000 tiles (100,000 map views⁴) jawg-terrain

¹ per month
² for our usage of ~ 1,340,000 tiles, prices without VAT
³ 50,000 map views⁴
⁴ 1 map view = 15 tiles
⁵ non-commercial and unrestricted public access only

vs. own rendering server

Community/free tile servers

Terms of use / permission to use needs to be checked.

World-wide

Language world-wide

Regional

Europe
Country

Overlays

world-wide
regional

nrenner avatar Dec 12 '18 17:12 nrenner

In any case, will there be any "free" alternative for our current usage? I mean, even osm.org tiles are quota-limited, aren' we already reaching the limit?

Are there any alternative but finding a sponsor or accepting donation and pay for it?

bagage avatar Dec 12 '18 20:12 bagage

This issue raises all kinds of questions, like: what are we doing here?

I also thought about the other servers. Ideally those would throttle or deny requests by referer when it gets too much, but I don't know if they can or do.

And when all maps become unusable, we would eventually need to resort to the "bring your own map" idea as proposed in #93 (in German; about integrating Mapsforge renderer like mapsforge-web, Mobac or mapsforge-tile-server). Or shut the online service down and make it easier to install and run it locally, including offline rendering.

My hope is that having many tile servers to choose from, will distribute the load among them.

Not sure about this yet, but we could also automatically put regional and localized maps on top of the layer switcher, depending on the browser language and the current map view. For example, a French-speaking user looking at a part of France would get the tiles from openstreetmap.fr by default and francetopo.fr as second base layer option.

While searching I found that others had this problem too and e.g. GPSies.com and Bikemap.net switched to only offer OpenCycleMap with their paid premium services.

I wonder if accepting sponsoring in exchange for a link to the sponsor would already make us commercial. Which would bring additional burdens like imprint requirement, possibility to get cease and desist letters from competitors and what-not. And at the moment I would feel uneasy about taking donations.

nrenner avatar Dec 13 '18 17:12 nrenner

Just a note about OpenMapTiles. As far as I understand so far, OpenMapTiles is a "standard" schema for storing data into layers into vector tiles. The data they provide is only here as a conveniency but you can easily build it from a PostgreSQL + PostGIS database (with an import of an OSM dump).

Qwant (Maps) recently opensourced their configuration for this, which can be an easy way to get this started: https://github.com/QwantResearch/qwantmaps. Importing of OSM data into the db is handled through https://github.com/QwantResearch/kartotherian_config/tree/master/import_data and the Kartotherian / Tilerator config will generate vector tiles following OpenMapTiles schema I think https://github.com/QwantResearch/kartotherian_config/tree/master/tilerator.

Phyks avatar Dec 23 '18 20:12 Phyks

What's the estimated bandwidth and compute requirements for running our own tile server? I'm happy to contribute resources (within reason).

leoluk avatar Dec 31 '18 22:12 leoluk

In case this could be useful, I started working on a (vector) style similar to the OpenCycleMap style: https://github.com/Phyks/cyclosm-basic-gl-style. So far, this would still require running your own tile server though (although this could change in the near future if the features I need for the style get integrated into OpenMapTiles).

A demo on a (limited) area is available here. Note that this is still WIP. :)

Phyks avatar Jan 16 '19 10:01 Phyks

@Phyks

The data they provide is only here as a conveniency but you can easily build it from a PostgreSQL + PostGIS database

I added the OpenMapTiles package download to show the requirements and costs of different approaches. It would allow to use a small server that just serves static vector tiles from the MBTiles file (SQLite DB).

The server requirements and costs for rendering OpenMapTiles from PostGIS are probably comparable to traditional raster tiles. For Leaflet, we would probably still want to render raster tiles with Tileserver GL.

Not sure about "easily": there doesn't seem to be a Tutorial for generating planet tiles? (it's their business) and following the Quickstart ("not optimal for a Planet rendering !!") seems to take > 34 days for the planet. They are using a cluster of 32 machines with 4 cores that takes 1 day (slide 46).

Using postserve (comment) for on-the-fly tile generation from PostGIS would allow to skip MBTiles generation though.

I gave tilemaker a quick try, with the idea to generate tiles from 5 degree extracts, corresponding to BRouter brf files:

osmium extract -b 5,45,10,50 -s smart --set-bounds germany-latest.osm.pbf -o E5_N45.osm.pbf

tilemaker --input E5_N45.osm.pbf --output E5_N45.mbtiles --config tilemaker/resources/config-openmaptiles.json --process tilemaker/resources/process-openmaptiles.lua

But I ran into an error (want to track further down) and a smaller file took longer than I hoped.

Qwant (Maps)

Thanks, I had seen the Qwant Maps/Kartotherian stack and my first impression was, that this is all a bit over the top for our purpose. I haven't found any server requirements or cost estimates.

https://github.com/Phyks/cyclosm-basic-gl-style

Thanks for sharing. Nice to see routes in a vector tile style.

nrenner avatar Jan 17 '19 17:01 nrenner

Thanks, I had seen the Qwant Maps/Kartotherian stack and my first impression was, that this is all a bit over the top for our purpose. I haven't found any server requirements or cost estimates.

Yes, however this stack is more documented and used in production for generating planet builds. They have a differential mode for easier daily update, see https://github.com/QwantResearch/qwantmaps#daily-osm-updates-. Still, I don't know much about their infrastructure and the power requirements :/

Phyks avatar Jan 17 '19 19:01 Phyks

When the data is loaded in PostgreSQL, we use tilerator to generate all the tiles from the zoom level 0 to 14 and store them in Cassandra.

https://github.com/QwantResearch/qwantmaps#tiles-data-

It seems they're using the same approach as OpenMapTiles.com, to pre-generate all tiles for the whole planet, which means 32 cluster servers working for one day.

nrenner avatar Jan 17 '19 21:01 nrenner

It seems they're using the same approach as OpenMapTiles.com, to pre-generate all tiles for the whole planet, which means 32 cluster servers working for one day.

Sure, but their script is working in diff mode. Then, I guess this "32 clusters servers working for one day" might be just a one-time cost. Not sure about how much tiles would be regenerated every day. In tilerator as well, they try to just generate what is actually needed https://github.com/QwantResearch/kartotherian_config/blob/master/import_data/tasks.py#L398-L406.

Note that their kartotherian setup can be used to generate tiles live (on request) from the database and store a cached copy in Cassandra, without any pregeneration (https://github.com/QwantResearch/kartotherian_config/blob/master/tilerator/sources.yaml#L77). I did not test the pre-generation setup yet actually, but the live generation from database is working fine for me.

Phyks avatar Jan 18 '19 07:01 Phyks

Then, I guess this "32 clusters servers working for one day" might be just a one-time cost.

I assume so, yes. Still sounds scary, would need to see what the actual cost on AWS or whatever would be. The original OSM2VectorTiles thesis says:

The cloud instances to render the world once costs around 1500 dollar.

Note that their kartotherian setup can be used to generate tiles live (on request) from the database and store a cached copy in Cassandra, without any pregeneration.

That's great then.

nrenner avatar Jan 18 '19 09:01 nrenner

Hi,

Just a quick update on the alternative render for bikes I was working on. In the end, generating vector tiles seemed to be difficult, especially since there are not many toolchains, infrastructure etc available in a non profit way for this. Moreover, most people eager to contribute are actually more familiar with CartoCSS and Mapnik rendering than vector techniques. Therefore, I started back in CartoCSS, close to the OSM.org render for instance and the style can be found at https://github.com/cyclosm/cyclosm-cartocss-style/. I serve a demo at https://tiles.phyks.me/#12/48.8588/2.3470 for some areas in France (Paris, Clermont-Ferrand and Lille surroundings).

I don't have the infrastructure to render tiles at large scale or provide a ready to use alternative, but this aims to be a FOSS alternative to OpenCycleMap. We did quite a lot of work on data integration, but this is still a WIP and we are lagging a bit on the graphic design side at the moment :/ This could be one extra rendering style to consider, but sadly not a ready to use solution.

Best,

Phyks avatar Mar 04 '19 23:03 Phyks