FiniteStateEntropy

FiniteStateEntropy copied to clipboard

a step in normalization of count

2

I am new here and still reading the source code. In this line, why normalizedCounter[s] = -1? In my opinion, -1 should be 1. Could you please explain it to...

BeeBreeze

NULL pointer dereference in BIT_reloadDStream()

7

Crashing line: 430: bitD->bitContainer = MEM_readLEST(bitD->ptr); in bitstream.h in the function BIT_reloadDStream() triggered by FSE_decompressU16() with the following code: ``` const uint8_t* casted2 = (uint8_t*)"Àâ(¢x(Kùÿÿcb¿\a"; uint16_t out[256]; size_t ret =...

JohanKnutzen

Corrupted block detected error reported in -hb mode

3

The following file produces the following error: `./programs/fse -bh ./badfile FSE : Finite State Entropy, 64-bits demo by Yann Collet (Nov 9 2018) !! Error decompressing block 986 of cSize...

michaelmaniscalco

FSE_compressU16() computes clipped data if dstCapacity different

1

Repro: ``` constexpr size_t inSize = 24; uint16_t in[inSize] = {0, 0, 3, 2, 0, 0, 0, 0, 314, 0, 0, 0, 0, 51, 50, 0, 0, 59, 22, 36,...

JohanKnutzen

question: FSE doesn't always compress better than huf - expected?

2

I can't share the data, but in short, it's 9108363519 bytes (~9GB) of almost-uncompressible data (IIRC it's the 9GB tail of a larger already-compressed stream). % ./fse -e ./almost-uncompressable Compressed...

adamdmoss

benchmark mode claims a decompression error on this data w/huf

5

``` % ./fse -h -b ./xxx FSE : Finite State Entropy, 64-bits demo by Yann Collet (Jul 17 2020) !! Error decompressing block 4 of cSize 18041 !! => (Corrupted...

adamdmoss

FSE_compress returns 0 even when maxDstSize is still equal to or slightly larger than final compression size

3

While adding more unit tests, I found a corner case when trying to compress a document with a `maxDstSize` which is exactly the expected compressed size (found by a previous...

KrzysFR

Error 39 : Decoding error : Destination buffer is too small

1

I tried diagnosis of the source code and using the clang-6.0 reports to find the issue for 10min, but wasn't lucky. My best guess is that the clang diagnosis is...

easterEodj

Architecture specific optimizations

6

Hello, I would like to know if it is possible to have ARM's SIMD (neon) routines to be added in huff0 and/or FSE encode/decode parts? That way, I can make...

codekatana

Comparison to arithmetic coding?

8

I know the wiki says that the performance is similar, but can we get a benchmark comparing processing time and compression ratio to know exactly how well it performs compared...

MarcusJohnson91