lz4-java
lz4-java copied to clipboard
lz4
LZ4 compression for Java
The [Intel Storage Acceleration Library](https://github.com/intel/isa-l) provides very fast zlib-compatible compression and also very fast CRC calculation. Speed ups from 3-6x are not uncommon in the decompression/compression department. Despite the name...
XXH3 and XXH3_128 are now stable since [XXHash release 0.8.0](https://github.com/Cyan4973/xxHash/releases/tag/v0.8.0). They perform even faster than XXH64 which is quite remarkable. It would be nice if these were available from lz4-java,...
The current LZ4 C lib supports priming the codec state with a dictionary (in a newer, nicer way). It would be nice to surface those features in this library. I'm...
Hello, I'm trying to decompress using "LZ4DecompressorWithLength.getDecompressedLength(input)" which is providing a huge byte size even for 1000 bytes of data. For e.g. If the Input Size is 2008 bytes then...
We are using version 1.5.1 and we've seen multiple instances where we have dead lock in LZ4Factory. Spark pipeline hangs and we end up killing it and restarting. Doesn't happen...
AFAIK, its not some "convention" or something for output streams and also lz4 doesnt use a checksum. From my tests it uses significant CPU and if its there for the...
I enabled Travis CI for arm64, ppc64le, and s390x by 58b50d2eed9d727878333a0dcf11e8729e0759f0 but found only s390x timed out (10 min) during `ant test`, so I disabled s390x for now by 8668c6e20cc153647b3b073f3151e03634adac23.
When it's ok to start requiring Java 9+ (maybe now is fine since Java 11 is the current LTS release?) we should implement `LZ4SafeUtils#commonBytes` using `Arrays#mismatch`. It yielded a good...
JDeps is reporting the usage of `sun.misc.Unsafe` (i.e. in `UnsafeUtils.java`) that is causing issues with our build process. Has there been any investigation into replacing this in the LZ4 library?...
When using this library in a highly concurrent setting - vis. Apache Spark - I'm seeing a lot of contention around [this](https://github.com/jpountz/lz4-java/blob/master/src/java/net/jpountz/util/Native.java#L66]) synchronisation block, specifically when called from [here](https://github.com/jpountz/lz4-java/blob/master/src/java/net/jpountz/lz4/LZ4Factory.java#L135) It...
