Overpass-API
Overpass-API copied to clipboard
drolbr
OSM_Element_Metadata doesn't initialize its variables
Recently I've switched my minute diffs from http://download.openstreetmap.fr/replication/ to http://download.geofabrik.de/europe/.
And while openstreetmap.fr provides all the metadata information, Geofabrik provides only version information (and no values for attributes like changeset, user, uid). Using such diffs for update_database results in database "corruption", as random values land in changeset.
As I've checked source code node_start, way_start and relation_start they create OSM_Element_Metadata object but changeset, user_id and user_name fields are not set if not provided in source. Object constructor does not initialize changeset (and I guess that constructor depends on implicit empty string initialization for user_name), though user_id is.
Looks like following change fixes problem.
Though looking around the code, it might be needed to initialize more variables (timestamp, version), but I do not know, how this might affect performance.
I believe you can also fetch diff files from Geofabrik which include metadata. Due to GDPR restrictions, you need a special download tool and authenticate using your osm user account: https://osm-internal.download.geofabrik.de/index.html.
You can also run Overpass without metadata by leaving out the "--meta" tag for your dispatcher process and the update_database / update_from_dir executables.
@mmd-osm thank you for the pointer. Still I think it might be worthwhile to include this change as it looked to me like data corruption at first and I lost confidence in my hardware and data.
I think a better fix would be to reject processing such minutely diffs in case you've started your dispatcher with --meta or --attic. After all, there isn't much point running your Overpass server in meta data mode and then try to load some files without any meta data.
As an extra bonus, you don't need additional hard disk space for completely useless meta data.
Superseded by https://github.com/drolbr/Overpass-API/commit/4a9cf9d5ce52374288df0d48868dba2be69872ea