worldcoin
[POP-2513] Make mask shares full length again
This is in preperation for the fragile bits effort for iris codes. Up until now, mask code shares have had the same imaginary and real part, which meant that we could reduce the storage and memory costs by only storing half and multiplying the results by 2. This PR extends them to the full length again, such that future iris codes may have different real and imaginary parts.
The DB and S3 loader are modified in a way that they extend half-sized mask codes to full ones if needed.
Also: DO NOT MERGE FOR NOW, this may need some sync for correct DB exporting in future.
Impact of merging this:
- Will increase memory requirements for InMemoryStore of IrisDB from 38.4kB to 51.2kB per iris code (a 33% increase over current).
- Processing time will increase slightly (mask dot products go from size 6400 to size 12800). For GPU, this time is relatively negligible compared to the remainder of the MPC protocol. For CPU, I do not have current breakdowns but I would also expect this to be not that significant.
- Will enable iris codes to have masks with different real/imaginary part in the future. The request shares currently sent via SNS are already in extended format.
@bgillesp @naure I tried to also handle changes in the HNSW variant, please take another look if I missed something