lz4_flex icon indicating copy to clipboard operation
lz4_flex copied to clipboard

Better documentation about what a "dictionary" is.

Open inodentry opened this issue 2 years ago • 1 comments

Looking at the documentation, I see that there are functions for compressing/decompressing "using an external dictionary". It is just a slice of bytes. What should those bytes be? How is it used by the algorithm?

I'm not super well versed in data compression theory and trying to learn...

My guess is that, since the algorithm works by finding back-references to previously-encountered data, the dictionary is just a bunch of bytes that will be treated as if they had come before the start of the actual data to compress? (to give the algorithm something it can refer to while it is still at the start of the compression sequence and hasn't encountered much "real" data yet)

Is my guess/assumption correct?

If so, would be nice if something about this gets added to the documentation...

inodentry avatar Mar 16 '22 13:03 inodentry

Sorry for the late reply, but yes that's exactly how the dictionary works.

PSeitz avatar Jul 31 '22 12:07 PSeitz