TurboPFor-Integer-Compression icon indicating copy to clipboard operation
TurboPFor-Integer-Compression copied to clipboard

Please document slop requirements

Open sesse opened this issue 4 years ago • 1 comments

Hi,

It seems that the PFor functions can read bytes past the end of the posting list, and write more output values than “num”. In particular, 32 values seem to be decoded at a time even if num == 1. This means the application must be careful to allow some slop to avoid crashes. Is the amount of required slop documented anywhere, ie., M extra bytes of input and space for N output values must always be available?

sesse avatar Oct 04 '20 16:10 sesse

some of it is "documented" in the maintainers answers here: https://github.com/powturbo/TurboPFor-Integer-Compression/issues/26

benrobby avatar Jan 18 '21 22:01 benrobby

I've made changes, so that there must be nor more reading/writing outside the i/o buffers without loosing any speed. More tests are necessarry for confirmation. There are now bound (ex. p4bound32) functions delivering the maximum output buffer size, required only in encoding. This can be used to allocate the output array buffer.

powturbo avatar Mar 23 '23 11:03 powturbo