TW
Multithreading is planned for after borg 2. The problem with approaches like the compressor internally implementing multithreading is that borg has a chunk size of typically 2MiB (and that is...
There is also #37 and #3500 for more details.
@Forza-tng I somehow suspect (for reasons I stated in my post above), that just giving that flag to zstd, isn't improving things much. Can you do a practical experiment, implementing...
I was curious, so I did that practical experiment (using a Apple M3 Pro CPU (ARM), macOS Sonoma, python 3.9.6, pyzstd 0.16.0, zstd 1.5.6): https://paste.thinkmo.de/E6NWYdQP#pyzstd-mt-test.py is there anything wrong with...
I am redoing the tests now with the pypi "zstd" package, which is a different implementation. zstd-1.5.5.1 package, everything else as above. workers == 0 means "autodetect" for this lib....
Considering that these test results do not show any significant speedup (but often even a significant slowdown), I guess this confirms my gut feeling that we do not have enough...
@justinsteven could you do a PR with a test for this (which fails with current code) and a fix for the locking code that makes that test succeed?
If it randomly fails, we could skip it by default. It would be useful nevertheless, just to check it works / to find if it doesn't.
From #netbsd: ``` 13:45 @racoon$ fixed ``` So, let's see when the fixed package is available from the mirror.
Note: the 0.8.2nb1 package from April still has the bug.