vasi
pixz isn't using all cores during decompression
I just tired comrpessing a bit apitrace file (13 GB), which got compressed to around 1GB. During compression all cores are used, but I noticed that during decompression, only a small portion of cores are used, and many remain idle. I'm running the test on Ryzen 7 1700X, so it has 8 cores and 18 threads available (hyperthreading).
Is decompression hard limited in how many cores it uses?
There shouldn't be any hard limit, but usually writing is slower than reading—so it's possible that's the bottleneck?
That could be the case, I'm using an HDD. I can get some SSD for a test.
On a side note, may be writing can be optimized with some buffering, to avoid the bottleneck? I.e. use more RAM for parallel writes in it, before writing out big chunk to disk in one go? That would minimize parallel disk access.
For level 8 and 9 compression, PIXZ does not give any advantage when decompressing. I have compared PIXZ with XZ.
