osm2pgrouting takes forever

[walpino]

Expected behavior and actual behavior it should do the import in x hours

Steps to reproduce the problem Connecting to the database connection success

Dropping tables... TABLE: hackways droped ... OK. TABLE: hackways_vertices_pgr droped ... OK. TABLE: hackpointsofinterest droped ... OK. TABLE: configuration droped ... OK. TABLE: osm_nodes droped ... OK. TABLE: osm_ways droped ... OK. TABLE: osm_relations droped ... OK.

Creating tables... TABLE: hackways_vertices_pgr created ... OK. TABLE: hackways created ... OK. TABLE: hackpointsofinterest created ... OK. TABLE: configuration created ... OK. TABLE: osm_nodes created ... OK. TABLE: osm_ways created ... OK. TABLE: osm_relations created ... OK. Opening configuration file: /usr/share/osm2pgrouting/mapconfig.xml Parsing configuration

Exporting configuration ...

Done Counting lines ... Done Opening data file: planet_8.4572,47.3346_8.5867,47.4007.osm total lines: 1542029 Parsing data Current osm_nodes: 400000 Final osm_nodes: 401143 Current osm_ways: 0 Final osm_ways: 65171

it stucks there for hours and hours and I dont see why and what it s doing the osm file is just a small one

Specifications like the version of pgRouting/PostGIS and PostgreSQL as well as Operating System cnetos SELECT version(); version PostgreSQL 9.6.6 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16), 64-bit (1 row)

SELECT postgis_full_version(); postgis_full_version POSTGIS="2.4.3 r16312" PGSQL="96" GEOS="3.6.2-CAPI-1.10.2 4d2925d6" PROJ="Rel. 4.9.3, 15 August 2016" GDAL="GDAL 1.11.4, released 2016/01/25" LIBXML="2.9.1" LIBJSON="0.11" RASTER (1 row)

SELECT pgr_version(); pgr_version (2.5.2,v2.5.2,60585f1f7,master,1.53.0) (1 row)

Use the commands:

SELECT version(); SELECT postgis_full_version(); SELECT pgr_version(); any idea where it is stucked? or what takes so long? the machine has 10 cpus and 50 gb of ram

[cayetanobv]

Hi @walpino . What is the osm2pgrouting version you are using? Could you paste the output of this command?

osm2pgrouting --version

Could you share your OSM dataset (or bounding box you are using to export OSM dataset)?

I'm seeing file total lines (aprox 1.5M) and it is very little for 50GB RAM. I'm processing right now without problems >4M lines with only 16GB RAM.

Thanks,

Cayetano

[znmeb]

I'm testing with the state of Oregon - the whole thing takes about 20 GB of RAM and the .osm file is 3.3 GB. I have a smaller region that I'm really interested in - the TriMet (local transit system) service area. That's only a 1.4 GB .osm file and only needs 8.6 GB of RAM to load into the database.

[cayetanobv]

Hi @znmeb ,

If you didn't removed metadata tags (not necessary for PgRouting) you can reduce a lot the size of your Oregon OSM file following these steps (you can get around half size of original file; 3.3GB to 1.7GB aprox):

• Download PBF file for entire Oregon (http://download.geofabrik.de/north-america/us/oregon.html). PBF is a binary format and it has a lower size (better for downloading operations).
• Remove unnecessary metadata tags from file with osmconvert (this tool is pretty fast parsing osm files). Example:

$ osmconvert output_data.osm.pbf --drop-author --drop-version --out-osm -o=output_data_reduc.osm

• Try again to run osm2pgrouting with this new file.

Thanks for using osm2pgrouting!

Cayetano

[znmeb]

@cayetanobv Thanks! That helped a bunch - I was doing the bounding box filtering with osmosis but osmconvert is faster and the resulting .osm output is only 678 megabytes. In my environment, the peak resident set size in osm2pgrouting is about 6.1 times the input size, which means I'll be a bit over 4 GB now.