zstd icon indicating copy to clipboard operation
zstd copied to clipboard

Allow Build-Time Exclusion of Compression Components

Open felixhandte opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. Zstandard has grown into a large tool that tries to serve a wide range of compression use cases. In order to achieve the best performance possible across this spectrum, Zstd ships a number of matchfinders optimized for different points in the speed-ratio curve (ZSTD_fast, ZSTD_dfast, ZSTD_greedy, ZSTD_lazy, ZSTD_lazy2, ZSTD_btlazy, and ZSTD_btopt). Each of those strategies are then instantiated many times, for different memory layouts (single-segment, split-segment, and dict-match-state) and for different minimum match lengths (4, 5, 6, or 7). All of these implementations end up producing a pretty fat binary.

Similarly, though less egregiously, the huffman encoder implements 4 different modes (1X1, 1X2, 4X1, 4X2).

A number of use cases have appeared which want to use Zstandard but where binary size is a significant cost. E.g., busybox, mobile apps, embedded systems, browsers.

Describe the solution you'd like We should create tooling--a set of macros they can define--that allows integrators to select a subset of zstd's compression implementations to include. This would allow space-constrained integrators to select only the functionality they need and exclude the rest. I expect this can potentially allow a huge reduction in binary size (>75%?).

Zstd will need to be robust to these exclusions and gracefully degrade when configurations are requested which have not been compiled in.

felixhandte avatar Dec 20 '21 18:12 felixhandte