knowhere icon indicating copy to clipboard operation
knowhere copied to clipboard

too narrow constraints in a unit test?

Open alexanderguzhva opened this issue 7 months ago • 5 comments

I've noticed a failure on a master branch (20caafd65e5d787188ea6a15b8a4b91f751267f4) in Knowhere unit tests. UT were compiled with clang-17 on an AVX-512 machine. gcc-9 shows no problems.

It seems like a 'noise' in floating-point operations.

I0703 14:11:50.064527  3057 hnsw.cc:139] [KNOWHERE][operator()][knowhere_build6] HNSW build progress: 80%
I0703 14:11:50.069895  3056 hnsw.cc:139] [KNOWHERE][operator()][knowhere_build5] HNSW build progress: 90%
I0703 14:11:50.074909  3035 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][knowhere_tests] Building HNSW cost: graph build (42.129970 ms)
I0703 14:11:50.075062  3035 hnsw.cc:150] [KNOWHERE][Add][knowhere_tests] there are 0 points can not be reached
I0703 14:11:50.075078  3035 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][knowhere_tests] Building HNSW cost: graph repair (0.176110 ms)
I0703 14:11:50.075088  3035 hnsw.cc:160] [KNOWHERE][Add][knowhere_tests] HNSW built with #points num:1000 #M:128 #max level:1 #ef_construction:200 #dim:128
I0703 14:11:50.075103  3035 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][knowhere_tests] Build index: done (42.780289 ms)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
knowhere_tests is a Catch2 v3.3.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
Test Iterator Mem Index With Float Vector
  Test Search with Bitset using iterator insufficient results
-------------------------------------------------------------------------------
/home/ubuntu/zilliz/knowhere_ut/knowhere/tests/ut/test_iterator.cc:205
...............................................................................

/home/ubuntu/zilliz/knowhere_ut/knowhere/tests/ut/test_iterator.cc:238: FAILED:
  REQUIRE( recall > kKnnRecallThreshold )
with expansion:
  0.8f > 0.8f
with messages:
  name := "HNSW_SQ8"
  cfg_json := "{"M":128,"dim":128,"ef":64,"efConstruction":200,"k":5,
  "metric_type":"COSINE"}"

alexanderguzhva avatar Jul 03 '24 14:07 alexanderguzhva