luceneutil icon indicating copy to clipboard operation
luceneutil copied to clipboard

Create tooling to measure quantization error

Open shubhamvishu opened this issue 1 year ago • 2 comments

@mikemccand mentioned it would be I'd good to have a tooling to measure/benchmark the overall quantization error for a given index or vector dataset. It could go over each float to check the induced error converting float -> quantized half/full byte -> float (rescaled value) the induced error and report the average induced quantization error/noise. Maybe also also playing with the confidenceInterval values and plot some nice looking charts?

shubhamvishu avatar Oct 24 '24 15:10 shubhamvishu

KnnGraphTester.java already reports some interesting stats about the HNSW graph ... maybe it could also measure/aggregate/report the quantization error?

mikemccand avatar Nov 02 '24 18:11 mikemccand

@mikemccand Indeed it would be useful to include this in the KnnGraphTester output itself. I'll put out a PR for this(sometime this week).

shubhamvishu avatar Nov 05 '24 08:11 shubhamvishu