Add replication file generation and replication from file [WIP]
This adds a facility to generate replication files in the "classical" OSM directory layout style and a facility to consume such files from a remote location via http.
New options:
Replication files generation:
-replication-dir dir directory for replication files (default '.')
-generate-files generate replication files while updating from Nominatim
-replicate generate a single replication file without starting the server
Consuming replication files: -replication-url url url of base directory for file based replication (default none) -replication-interval number interval between trying to read replication files in minutes (default 60)
This is work in progress and will need some testing irl.
I've played with hourly planet diffs. The resulting files are about half the size of the original diff, which is good.
Looking at the format, we need to get rid of any traces of Nominatim place IDs as they are tied to instances and it must be possible to create a continuous stream of diffs even when the producing instance changes. The ids should be fairly easy to replace. A string of <osm type><osm id><class> uniquely identifies a document. The other fields that contain place ids could preferably be just left out of the dump. They are only intermediate values that are needed to compute the address fields but they are not used in the ES document (although they seem to be imported).