zstd
zstd copied to clipboard
seekable_format/examples/parallel_compression.c is not parallel
Describe the bug The parallel compression example does not seem to be working in parallel. It runs on a single thread only, regardless of how many threads are requested.
To Reproduce
- Build the example parallel_compression
- Run:
./parallel_compression data.txt $((2*1024*1024)) 4data.txt can be any sufficiently large file so that compression takes a handful of seconds. In my case, this is ASCII data, 2.8 GB. - While the compression is running, look at e.g. top
- Just a single thread is used. This is usually slower than the single threaded seekable_compression,
Expected behavior
Parallel execution should use multiple cores and be faster than single threaded.
The full zstd executable works as expected; given the -T option it is faster than without and saturates all cores.
Also, I'd love for --seekable to become a standard option for zstd, but that should probably be a separate issue.
Desktop:
- OS: Linux
- Version Ubuntu 22.04
- Compiler: gcc 11.4
- Flags: -O3
- System: Quad-core Intel Core i7-1165G7
- Build system Makefile
- Built from git version: commit 559762da12f54712d44f619098aa4a7e7bc5727b, Mar 14 2024