luceneutil
luceneutil copied to clipboard
Create tooling to measure quantization error
@mikemccand mentioned it would be I'd good to have a tooling to measure/benchmark the overall quantization error for a given index or vector dataset. It could go over each float to check the induced error converting float -> quantized half/full byte -> float (rescaled value) the induced error and report the average induced quantization error/noise. Maybe also also playing with the confidenceInterval values and plot some nice looking charts?
KnnGraphTester.java already reports some interesting stats about the HNSW graph ... maybe it could also measure/aggregate/report the quantization error?
@mikemccand Indeed it would be useful to include this in the KnnGraphTester output itself. I'll put out a PR for this(sometime this week).