Problem with merged output

[djmakwana]

What version of osmium-tool are you using?

docker run --rm -it -v .:/data stationa/osmium-tool --version
osmium version 1.9.1 (v1.9.1)
libosmium version 2.14.2
Copyright (C) 2013-2018  Jochen Topf <[email protected]>
License: GNU GENERAL PUBLIC LICENSE Version 3 <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

What operating system version are you using?

https://hub.docker.com/r/stationa/osmium-tool

Tell us something about your system

The docker host is running 128GB RAM, 24 Cores

What did you do exactly?

I am trying to merge the two different region files downloaded from https://download.geofabrik.de/south-america-latest.osm.pbf and https://download.geofabrik.de/australia-oceania-latest.osm.pbf

The exact command is as follows

docker run --rm -it -v .:/data stationa/osmium-tool merge --progress /data/australia-oceania-latest.osm.pbf /data/south-america-latest.osm.pbf -o /data/au_sa.osm.pbf

Then when I tried to import the au_sa.osm.pbf file in the https://github.com/Overv/openstreetmap-tile-server using their import command, it fails with an error (see below). I have also created an issue (https://github.com/Overv/openstreetmap-tile-server/issues/404) in that repository as I am not sure where the problem lies.

Error

ERROR: Input data is not ordered: relation id 2202162 appears more than once.

Here's my docker comman in the docker-compose.yml

version: '3.7'

services:
  map_importer:
    image: overv/openstreetmap-tile-server
    volumes:
      - ./data/osm-data:/data/database/
      - ./data/osm-tiles:/data/tiles/
      - /mnt/data/osm_map_tiles/au_sa.osm.pbf:/data/region.osm.pbf
    ports:
      - "9090:80"
    command: "import"
    environment:
      - THREADS=12
      - OSM2PGSQL_EXTRA_ARGS=-C 4096

What did you expect to happen?

As per the documentation of the merge command, I did not expect the merged file to contain duplicate ids and it is supposed to be sorted.

What did happen instead?

Error

ERROR: Input data is not ordered: relation id 2202162 appears more than once.

What did you do to try analyzing the problem?

I am new to this area and am looking for some suggestions.

[joto]

The manual has this to say: "If there are different versions of the same object in the input files, all versions will appear in the output... Do not use this command to merge non-history files with data from different points in time. It will not work correctly."

So most likely the two files you downloaded and merged were from different points in time. Maybe you downloaded one on one day and the other the next day.

[djmakwana]

Thank you for the clue. I am downloading all new files today and will report back when I try out my small experiment.

[ImreSamu]

@djmakwana:

I am downloading all new files today

There is a small chance, but due to time zone differences, the "today" you mentioned might coincide with the data update time. In such cases, it might be useful not to use the '-latest' option.

In the "raw directory index", you will find files tagged with dates in the format YYMMDD. For example, 231212 represents the date 2023-12-12.

https://download.geofabrik.de/australia-oceania-231212.osm.pbf
https://download.geofabrik.de/south-america-231212.osm.pbf

[djmakwana]

Thanks you