Tile generation using --store is singlethreaded

[lokkelvin2]

Using --store to generate the asia extract seems to run in single-threaded mode. This is after reading the pbf and at the zoom xx: writing tile xxx of xxx stage. My machine runs a dual socket Xeon 8168 with 192gb of RAM and the --store cache is 315gb on a nvme ssd. I believe that the computation at this stage should be multithreaded. However, cpu usage is only on 1 out of 96 cores and my disk i/o is basically 0%.

I am using the CI generated Windows artifact from commit 4053d523a31596d0156cd0458d17c8c45dbbbaff .

Any ideas what I could be doing wrong?

[kleunen]

It should be multithreaded at this phase. Also reading the pbf is multi-threaded.

Maybe try to start the process with explicit specify the number of threads with: --threads 96

[lokkelvin2]

Thanks for the advice. I just retried it on my xeon machine and no dice, the CPU utilisation is still in the single digits. Strangely enough, the same binary runs multithreaded on a Ryzen 3600 pc when started without --store. I'll do some more investigation on the Xeon machine to see where the bottleneck is.

[kleunen]

yes, if you could investigate what the problem might be, that would be helpful, because I honestly have no clue what could be the problem. It is the windows executable, correct ?

[kleunen]

Without --store it does run fine ?

[lokkelvin2]

After some testing, I settled with generating the entire continent by merging countries together using --merge and running the commands sequentially with a batch script. This way, I can see some progress in the tile generation, and it runs multithreaded without using --store on my xeon machine.

I'm still not sure what caused the singlethreaded behaviour when using --store.