python-lz4
Missing support for dictionary compression (unstable, non-public API)
I'm looking into implementing this but hit an upstream roadblock: https://github.com/lz4/lz4/issues/791. The frame dictionary APIs are still considered unstable (despite their age) and are only available with statically linked copies of lz4. Today, python-lz4 supports both but prefers the dynamically-linked system copy of lz4 if available.
We can fail at runtime still if we are using a dynamically linked copy, but how lz4 was built into the package is pretty opaque to the user.
Yup - we won't be adding this to the python bindings until the API is stable and public.
@jonathanunderwood-- do you think having this as experimental only and forcing static links for experimental builds is a good middle ground? This seems like it only isn't stable because there hasn't been a major lz4 release in the last three years.
@jonathanunderwood-- do you think having this as experimental only and forcing static links for experimental builds is a good middle ground? This seems like it only isn't stable because there hasn't been a major lz4 release in the last three years.
I'd consider it if someone submitted a PR and the added complexity in setup.py etc wasn't high and the submitter was prepared to stick around and support it. I have very limited bandwidth to support this package, and this really isn't a high priority compared to the other stuff that is outstanding (fixing/rewriting the frame bindings, Aarch64 builds, type hinting). But honestly, before that, I'd need to see convincing numbers demonstrating it's actually worth it.
Note also issue #236 means we'll be making changes to setup.py and how compilers and flags are set up.
Looks like dictionaries are now fully enabled in the LZ4 https://github.com/lz4/lz4/releases/tag/v1.10.0
