ann-benchmarks
ann-benchmarks copied to clipboard
Variability in test size
I’m trying to understand the variability in the Test Size of the datasets provided and if that has an impact on the QPS. The test sizes for datasets listed vary from 500 to 50,000. I had a few of questions:
- Is there any reason for the test size selection?
- Will smaller test size impact recall?
- In case we iterate more than once on this dataset for querying, i.e.
run_count > 1
, databases might have cached the results from earlier runs. In which case the impact on smaller test size should be more pronounced.