Allow multiple source in osm2pgsql-replication

[pinaraf]

Instead of assuming the replication_status table can contain only one source, make it possible to add multiple sources, and iterate on the table when running update. When replicating for instance Europe from geofabrik, if you want the whole french territories, this is a must-have : the europe.osm.pbf file does not include french territories like Réunion, Guyane or Guadeloupe. And likewise, their replication stream will not include these territories.

[pinaraf]

I've tested this successfully, so I'm marking it ready for review, but there is one pending question in the change: should I add an osm2pgsql-replication drop command, that would remove one source or all sources of replication?

[lonvia]

I'm still undecided what to do with this. I certainly see the demand for importing/updating from multiple extracts ad the changes in this PR are simple enough and look okay. The problem is with the details. Updating multiple extracts only works when the diffs are all ultimately from the same state of the planet. It happens to work with the Geofabrik extracts if the diffs are all from the same day. So there are all kinds of corner cases which have a way of making the code more and more complicated.

So the better approach might be here to have a geofabrik replication script that takes care of proper synchronisation.

[pinaraf]

So the better approach might be here to have a geofabrik replication script that takes care of proper synchronisation.

I thought about it, but I did not see the issue as long as imports are from distinct areas. What issue do you see with having diffs from different states?

[joto]

If you have data from adjoining areas there will be some overlap. There can also be overlap from relation members even if the areas aren't that close. And if you get different versions of the same object from the different input files, all sorts of problems can result.

[lonvia]

I'm closing this here. I'd rather see support for this on the pyosmium side to make sure it is implemented properly once and also works with the other pyosmium update scripts. I've opened https://github.com/osmcode/pyosmium/issues/214.