VectorDBBench
VectorDBBench copied to clipboard
vectordbbench should detect the index mis-match
Reproduce steps:
- create HNSW-SQ index with following command:
vectordbbench milvushnswsq --uri "http://10.220.56.72:19530" --case-type Performance1024D10M --drop-old --m 16 --ef-construction 128 --ef-search 100 --refine false --refine-type fp32 --refine-k 100 --sq-type sq8
- retest QPS as HNSW index type with following command:
vectordbbench milvushnsw --uri "http://10.220.56.72:19530" --case-type Performance1024D10M --skip-drop-old --skip-load --m 16 --ef-construction 128 --ef-search 100
vectordbbench should report index type mis-match, but not
@cydrain To clarify, vdbbench is not designed to recognize changes like "index_type mismatch."
The milvushnswsq and milvushnsw are both implementations of the Milvus client interface. For vdbbench, every client is a black box. Its role is to call standard functions like init, insert, and search and measure the performance.
It has no knowledge of the internal state, parameters, or configuration changes (such as index type) that occur within the client during these operations.