rii
rii copied to clipboard
rii.py Ks assert not consistent with nanopq's Ks assert
In rii.py line 35 there is an assert for Ks to check if it is less than 256 (which should be 2**32), but in nanopq pq.py line 37 similar assert checks to see if Ks is less than 2**32 (which is correct IMHO). So when passing in a nanopq to rii it throws AssertionError if the Ks is larger than 256. I am using a very large dataset 6.7 million rows. So my Ks is set 2**12 = 4096 and nanopq works as expected, but rii throws the AssertionError.
This behavior is expected.
I designed nanopq
with a maximum flexibility. So you can try 256<Ks
.
In rii
, each vector must be encoded to a set of 8-bit unsigned chars, meaning that Ks=256
. See flattened_codes
: https://github.com/matsui528/rii/blob/main/src/rii.h#L81
I would suggest using higher M
instead of higher Ks
in your use case. For example, let's say your vector is 120-dim and you encode it by M=4
with Ks=2**12
. resulted in 4 * 12 = 48 bit/code
. Then you can encode it by M=6
with Ks=2**8
, resulted in 6 * 8 = 48 bit/code
.