VectorDBBench icon indicating copy to clipboard operation
VectorDBBench copied to clipboard

vectordbbench should detect the index mis-match

Open cydrain opened this issue 2 months ago • 1 comments

Reproduce steps:

  1. create HNSW-SQ index with following command:
vectordbbench milvushnswsq   --uri "http://10.220.56.72:19530"   --case-type Performance1024D10M  --drop-old   --m 16   --ef-construction 128   --ef-search 100  --refine false  --refine-type fp32  --refine-k 100  --sq-type sq8
  1. retest QPS as HNSW index type with following command:
vectordbbench milvushnsw   --uri "http://10.220.56.72:19530"   --case-type Performance1024D10M  --skip-drop-old  --skip-load  --m 16   --ef-construction 128   --ef-search 100

vectordbbench should report index type mis-match, but not

cydrain avatar Nov 08 '25 16:11 cydrain

@cydrain To clarify, vdbbench is not designed to recognize changes like "index_type mismatch."

The milvushnswsq and milvushnsw are both implementations of the Milvus client interface. For vdbbench, every client is a black box. Its role is to call standard functions like init, insert, and search and measure the performance.

It has no knowledge of the internal state, parameters, or configuration changes (such as index type) that occur within the client during these operations.

alwayslove2013 avatar Nov 25 '25 10:11 alwayslove2013