osmpbf icon indicating copy to clipboard operation
osmpbf copied to clipboard

Add lower level API to more control over parallel execution

Open bjornharrtell opened this issue 3 years ago • 2 comments

Would be good to be able to get a ParallelIterator directly for cases where more control is needed.

bjornharrtell avatar Sep 19 '22 18:09 bjornharrtell

I agree and I will try to see if it is feasible.

b-r-u avatar Sep 21 '22 23:09 b-r-u

Hi, just used the code and I concur, when reading the nodes of a huge pbf file, it would be nice to have a number of parallel reads spread by the number of cores so that we can accumulate data in a per-thread container, then at the end use a mutex to merge them. Instead of locking a common container for every node insertion ( kills the performance).

Do you think it's achievable?

Thanks, great lib btw

ThomasCartier avatar May 19 '24 20:05 ThomasCartier