Efficient-Compression-Tool icon indicating copy to clipboard operation
Efficient-Compression-Tool copied to clipboard

Per file and per block multithreading don't work correctly

Open K0-RR opened this issue 1 year ago • 3 comments

ECT only uses 12,5% of my CPU. I have 16 threads so I used --mt-deflate=16 --mt-file=16 but looking at the system monitor only two threads are being utilized to 100% and the rest are idling.

K0-RR avatar Oct 22 '23 13:10 K0-RR

ECT uses multithreading, but there are limitations to how many threads it can use based on the input data. --mt-deflate uses several threads to compress the different blocks in a file. If the block splitting algorithm divides the data into n blocks, up to n threads can be used to compress the data as the data can't be divided arbitrarily without making the compression ratio worse. --mt-file compresses several files in parallel – if there are n files up to n threads will be used at once. Use care when using both options at once – if you are compressing several larger files with these options you might end up with 16^2=256 threads at once, which will slow down the overall system.

fhanau avatar Oct 23 '23 00:10 fhanau

@fhanau does ect use multithreading by default or do I have to pass mt arguments?

fideliochan avatar Jun 16 '24 21:06 fideliochan

multithreading is off-by-default. Since there are two different multithreading modes, it is non-trivial to define a good default multithreading setting.

fhanau avatar Jun 18 '24 21:06 fhanau