osmdata icon indicating copy to clipboard operation
osmdata copied to clipboard

Speed up fill_overpass_data

Open FlxPo opened this issue 2 years ago • 1 comments

I had performance issues with osmdata_sc used directly on OSM data (by providing a local .osm file), with medium to large files (30 Mo to 300 Mo, extracted with osmextract + osmium).

I think I found a way to speed things up : fill_overpass_data is loading the OSM xml data and then casting it as character, which appears to be very slow. I simply replaced doc <- as.character(doc) by doc <- readChar (doc, file.info (doc)$size) : on my machine, a quick test with a 50 Mo .osm file gives an execution time of 4s instead of 80s.

I'll make a PR with the proposed changes.

FlxPo avatar Nov 02 '21 13:11 FlxPo

Thanks @FlxPo - that looks like a very important improvement indeed! I'll check it out asap and get back to you.

mpadge avatar Nov 02 '21 13:11 mpadge