lucene icon indicating copy to clipboard operation
lucene copied to clipboard

Enforce fallback support for float vector retrieval in quantized KNN vector formats

Open Pulkitg64 opened this issue 1 month ago • 1 comments

Description

In PR #14792, we added fallback support to Lucene99ScalarQuantizedVectorReader to retrieve float vectors when full-precision vectors are not present in the index. However, this fallback support was removed by mistake during the OSQ implementation as part of the Lucene104 codec, requiring us to re-add it in PR #15415.

To ensure such support is enforced in future codecs, we need a mechanism to detect when it's missing. This PR addresses this by moving the relevant test case to BaseKnnVectorsFormatTestCase, ensuring that all subclasses must implement this functionality. The test will fail for any new codec that lacks the required fallback support.

Pulkitg64 avatar Dec 05 '25 10:12 Pulkitg64

Can we call it something other than quantizedFormat? There are may formats that I would call quantized that would return false here (e.g. all HNSW ones right?).

Maybe something like "supportsFloatVectorFallback()"?

Makes sense. Changed in next revision.

Pulkitg64 avatar Dec 09 '25 05:12 Pulkitg64