rii icon indicating copy to clipboard operation
rii copied to clipboard

rii.py Ks assert not consistent with nanopq's Ks assert

Open ashleyabraham opened this issue 2 years ago • 1 comments

In rii.py line 35 there is an assert for Ks to check if it is less than 256 (which should be 2**32), but in nanopq pq.py line 37 similar assert checks to see if Ks is less than 2**32 (which is correct IMHO). So when passing in a nanopq to rii it throws AssertionError if the Ks is larger than 256. I am using a very large dataset 6.7 million rows. So my Ks is set 2**12 = 4096 and nanopq works as expected, but rii throws the AssertionError.

ashleyabraham avatar Mar 12 '22 06:03 ashleyabraham

This behavior is expected.

I designed nanopq with a maximum flexibility. So you can try 256<Ks.

In rii, each vector must be encoded to a set of 8-bit unsigned chars, meaning that Ks=256. See flattened_codes: https://github.com/matsui528/rii/blob/main/src/rii.h#L81

I would suggest using higher M instead of higher Ks in your use case. For example, let's say your vector is 120-dim and you encode it by M=4 with Ks=2**12. resulted in 4 * 12 = 48 bit/code. Then you can encode it by M=6 with Ks=2**8, resulted in 6 * 8 = 48 bit/code.

matsui528 avatar Aug 06 '22 03:08 matsui528