knowhere
knowhere copied to clipboard
Published
20 hours ago •
zilliztech
zilliztech
[Enhancement] use std::visit to replace std::get_if
trafficstars
get_if is not very readable, we can use visit instead
issue: https://github.com/zilliztech/knowhere/issues/630
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Light-City
To complete the pull request process, please assign hhy3 after the PR has been reviewed.
You can assign the PR to them by writing /assign @hhy3 in a comment when ready.
The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
Welcome @Light-City! It looks like this is your first PR to zilliztech/knowhere 🎉
@Light-City 🔍 Important: PR Classification Needed!
For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:
- If you're fixing a bug, label it as kind/bug.
- For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
- Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
- Adjusting APIs or changing functionality? Go with kind/feature.
For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #
Thanks for your efforts and contribution to the community!.
![mergify[bot] avatar](https://avatars.githubusercontent.com/in/10562?v=4 "Published by a pub.dev verified publisher"){kind=small}
@Light-City Thanks for contributing. Plz use commit -s to add signature in the commit. Also, plz fix the CI issue before checking in
@Light-City Thanks for contributing. Plz use
commit -sto add signature in the commit. Also, plz fix the CI issue before checking in
I can't see any errors in CI, and I don't know how to fix it. Is it because I don't have permission? In addition, the local unit test passed.
there is an issue #686 , we're looking into it @Light-City Thanks for spotting one!
there is an issue #686 , we're looking into it @Light-City Thanks for spotting one!
fix this error commit is : https://github.com/zilliztech/knowhere/pull/631/commits/b9ee39fabd8981336ad8ea81b11082194d2edac2
Hi @Light-City There is still a clang-format error. And plz squash your commits into 1.
Thanks
Details
- Regarding clang-format issues, I used
git ls-files | grep -E '\.(cpp|h|cu|cuh)$' | xargs clang-format -i locallyand found the following diffs. These files do not seem to be introduced by my pr and have not been modified.- Commit rebase is 1 and has been completed
- ut in CI crashed and I can't see the reason
修改： benchmark/benchmark_base.h 修改： include/knowhere/config.h 修改： include/knowhere/heap.h 修改： thirdparty/DiskANN/include/diskann/aio_context_pool.h 修改： thirdparty/DiskANN/include/diskann/aligned_file_reader.h 修改： thirdparty/DiskANN/include/diskann/aux_utils.h 修改： thirdparty/DiskANN/include/diskann/index.h 修改： thirdparty/DiskANN/include/diskann/linux_aligned_file_reader.h 修改： thirdparty/DiskANN/include/diskann/logger_impl.h 修改： thirdparty/DiskANN/include/diskann/math_utils.h 修改： thirdparty/DiskANN/include/diskann/memory_mapper.h 修改： thirdparty/DiskANN/include/diskann/neighbor.h 修改： thirdparty/DiskANN/include/diskann/pq_flash_index.h 修改： thirdparty/DiskANN/include/diskann/pq_table.h 修改： thirdparty/DiskANN/include/diskann/utils.h 修改： thirdparty/DiskANN/include/tsl/robin_hash.h 修改： thirdparty/DiskANN/src/aux_utils.cpp 修改： thirdparty/DiskANN/src/index.cpp 修改： thirdparty/DiskANN/src/linux_aligned_file_reader.cpp 修改： thirdparty/DiskANN/src/math_utils.cpp 修改： thirdparty/DiskANN/src/partition_and_pq.cpp 修改： thirdparty/DiskANN/src/pq_flash_index.cpp 修改： thirdparty/faiss/faiss/Clustering.cpp 修改： thirdparty/faiss/faiss/Clustering.h 修改： thirdparty/faiss/faiss/FaissHook.h 修改： thirdparty/faiss/faiss/IVFlib.cpp 修改： thirdparty/faiss/faiss/IndexBinaryFlat.cpp 修改： thirdparty/faiss/faiss/IndexBinaryIVF.cpp 修改： thirdparty/faiss/faiss/IndexBinaryIVF.h 修改： thirdparty/faiss/faiss/IndexFlat.cpp 修改： thirdparty/faiss/faiss/IndexFlat.h 修改： thirdparty/faiss/faiss/IndexFlatElkan.cpp 修改： thirdparty/faiss/faiss/IndexFlatElkan.h 修改： thirdparty/faiss/faiss/IndexIVF.cpp 修改： thirdparty/faiss/faiss/IndexIVF.h 修改： thirdparty/faiss/faiss/IndexIVFFlat.cpp 修改： thirdparty/faiss/faiss/IndexIVFFlat.h 修改： thirdparty/faiss/faiss/IndexIVFPQ.cpp 修改： thirdparty/faiss/faiss/IndexIVFPQFastScan.cpp 修改： thirdparty/faiss/faiss/IndexIVFScalarQuantizerCC.cpp 修改： thirdparty/faiss/faiss/IndexIVFSpectralHash.cpp 修改： thirdparty/faiss/faiss/IndexLSH.cpp 修改： thirdparty/faiss/faiss/IndexPQ.cpp 修改： thirdparty/faiss/faiss/IndexScaNN.cpp 修改： thirdparty/faiss/faiss/IndexScaNN.h 修改： thirdparty/faiss/faiss/IndexScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/IndexShardsIVF.cpp 修改： thirdparty/faiss/faiss/MetricType.h 修改： thirdparty/faiss/faiss/clone_index.h 修改： thirdparty/faiss/faiss/gpu/GpuDistance.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndex.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexFlat.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexFlat.h 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVF.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFFlat.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFPQ.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFScalarQuantizer.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexThreadSafe.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFAppend.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFBase.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFFlatScan.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFInterleaved.cuh 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtils.cuh 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtilsSelect1.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtilsSelect2.cu 修改： thirdparty/faiss/faiss/gpu/impl/L2Select.cu 修改： thirdparty/faiss/faiss/gpu/perf/PerfSelect.cu 修改： thirdparty/faiss/faiss/gpu/test/TestGpuIndexIVFScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/gpu/utils/BlockSelectImpl.cuh 修改： thirdparty/faiss/faiss/gpu/utils/Tensor.cuh 修改： thirdparty/faiss/faiss/impl/RHNSW.cpp 修改： thirdparty/faiss/faiss/impl/RHNSW.h 修改： thirdparty/faiss/faiss/impl/ResultHandler.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizer.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerCodec_neon.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_avx.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_avx512.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_neon.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerOp.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerScanner.h 修改： thirdparty/faiss/faiss/impl/index_read.cpp 修改： thirdparty/faiss/faiss/impl/index_write.cpp 修改： thirdparty/faiss/faiss/impl/pq4_fast_scan.cpp 修改： thirdparty/faiss/faiss/impl/pq4_fast_scan_search_qbs.cpp 修改： thirdparty/faiss/faiss/impl/simd_result_handlers.h 修改： thirdparty/faiss/faiss/index_io.h 修改： thirdparty/faiss/faiss/invlists/DirectMap.cpp 修改： thirdparty/faiss/faiss/invlists/DirectMap.h 修改： thirdparty/faiss/faiss/invlists/InvertedLists.cpp 修改： thirdparty/faiss/faiss/invlists/InvertedLists.h 修改： thirdparty/faiss/faiss/utils/binary_distances.cpp 修改： thirdparty/faiss/faiss/utils/bit_table.cpp 修改： thirdparty/faiss/faiss/utils/data_backup_file.cpp 修改： thirdparty/faiss/faiss/utils/data_backup_file.h 修改： thirdparty/faiss/faiss/utils/distances.cpp 修改： thirdparty/faiss/faiss/utils/distances.h 修改： thirdparty/faiss/faiss/utils/distances_if.h 修改： thirdparty/faiss/faiss/utils/partitioning_avx2.cpp 修改： thirdparty/faiss/faiss/utils/structure-inl.h 修改： thirdparty/faiss/tests/test_dealloc_invlists.cpp 修改： thirdparty/faiss/tests/test_distances_if.cpp 修改： thirdparty/faiss/tests/test_ivf_index.cpp 修改： thirdparty/faiss/tests/test_mem_leak.cpp 修改： thirdparty/hnswlib/examples/searchKnnCloserFirst_test.cpp 修改： thirdparty/hnswlib/examples/updates_test.cpp 修改： thirdparty/hnswlib/hnswlib/hnswlib.h 修改： thirdparty/hnswlib/hnswlib/space_hamming.h 修改： thirdparty/hnswlib/hnswlib/space_jaccard.h 修改： thirdparty/hnswlib/hnswlib/visited_list_pool.h 修改： thirdparty/hnswlib/main.cpp 修改： thirdparty/hnswlib/python_bindings/bindings.cpp 修改： thirdparty/hnswlib/sift_1b.cpp 修改： thirdparty/hnswlib/sift_test.cpp
@liliu-z
Details
- Regarding clang-format issues, I used
git ls-files | grep -E '\.(cpp|h|cu|cuh)$' | xargs clang-format -i locallyand found the following diffs. These files do not seem to be introduced by my pr and have not been modified.- Commit rebase is 1 and has been completed
- ut in CI crashed and I can't see the reason
修改： benchmark/benchmark_base.h 修改： include/knowhere/config.h 修改： include/knowhere/heap.h 修改： thirdparty/DiskANN/include/diskann/aio_context_pool.h 修改： thirdparty/DiskANN/include/diskann/aligned_file_reader.h 修改： thirdparty/DiskANN/include/diskann/aux_utils.h 修改： thirdparty/DiskANN/include/diskann/index.h 修改： thirdparty/DiskANN/include/diskann/linux_aligned_file_reader.h 修改： thirdparty/DiskANN/include/diskann/logger_impl.h 修改： thirdparty/DiskANN/include/diskann/math_utils.h 修改： thirdparty/DiskANN/include/diskann/memory_mapper.h 修改： thirdparty/DiskANN/include/diskann/neighbor.h 修改： thirdparty/DiskANN/include/diskann/pq_flash_index.h 修改： thirdparty/DiskANN/include/diskann/pq_table.h 修改： thirdparty/DiskANN/include/diskann/utils.h 修改： thirdparty/DiskANN/include/tsl/robin_hash.h 修改： thirdparty/DiskANN/src/aux_utils.cpp 修改： thirdparty/DiskANN/src/index.cpp 修改： thirdparty/DiskANN/src/linux_aligned_file_reader.cpp 修改： thirdparty/DiskANN/src/math_utils.cpp 修改： thirdparty/DiskANN/src/partition_and_pq.cpp 修改： thirdparty/DiskANN/src/pq_flash_index.cpp 修改： thirdparty/faiss/faiss/Clustering.cpp 修改： thirdparty/faiss/faiss/Clustering.h 修改： thirdparty/faiss/faiss/FaissHook.h 修改： thirdparty/faiss/faiss/IVFlib.cpp 修改： thirdparty/faiss/faiss/IndexBinaryFlat.cpp 修改： thirdparty/faiss/faiss/IndexBinaryIVF.cpp 修改： thirdparty/faiss/faiss/IndexBinaryIVF.h 修改： thirdparty/faiss/faiss/IndexFlat.cpp 修改： thirdparty/faiss/faiss/IndexFlat.h 修改： thirdparty/faiss/faiss/IndexFlatElkan.cpp 修改： thirdparty/faiss/faiss/IndexFlatElkan.h 修改： thirdparty/faiss/faiss/IndexIVF.cpp 修改： thirdparty/faiss/faiss/IndexIVF.h 修改： thirdparty/faiss/faiss/IndexIVFFlat.cpp 修改： thirdparty/faiss/faiss/IndexIVFFlat.h 修改： thirdparty/faiss/faiss/IndexIVFPQ.cpp 修改： thirdparty/faiss/faiss/IndexIVFPQFastScan.cpp 修改： thirdparty/faiss/faiss/IndexIVFScalarQuantizerCC.cpp 修改： thirdparty/faiss/faiss/IndexIVFSpectralHash.cpp 修改： thirdparty/faiss/faiss/IndexLSH.cpp 修改： thirdparty/faiss/faiss/IndexPQ.cpp 修改： thirdparty/faiss/faiss/IndexScaNN.cpp 修改： thirdparty/faiss/faiss/IndexScaNN.h 修改： thirdparty/faiss/faiss/IndexScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/IndexShardsIVF.cpp 修改： thirdparty/faiss/faiss/MetricType.h 修改： thirdparty/faiss/faiss/clone_index.h 修改： thirdparty/faiss/faiss/gpu/GpuDistance.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndex.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexFlat.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexFlat.h 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVF.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFFlat.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFPQ.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexIVFScalarQuantizer.cu 修改： thirdparty/faiss/faiss/gpu/GpuIndexThreadSafe.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFAppend.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFBase.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFFlatScan.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFInterleaved.cuh 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtils.cuh 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtilsSelect1.cu 修改： thirdparty/faiss/faiss/gpu/impl/IVFUtilsSelect2.cu 修改： thirdparty/faiss/faiss/gpu/impl/L2Select.cu 修改： thirdparty/faiss/faiss/gpu/perf/PerfSelect.cu 修改： thirdparty/faiss/faiss/gpu/test/TestGpuIndexIVFScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/gpu/utils/BlockSelectImpl.cuh 修改： thirdparty/faiss/faiss/gpu/utils/Tensor.cuh 修改： thirdparty/faiss/faiss/impl/RHNSW.cpp 修改： thirdparty/faiss/faiss/impl/RHNSW.h 修改： thirdparty/faiss/faiss/impl/ResultHandler.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizer.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizer.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerCodec_neon.h 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_avx.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_avx512.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerDC_neon.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerOp.cpp 修改： thirdparty/faiss/faiss/impl/ScalarQuantizerScanner.h 修改： thirdparty/faiss/faiss/impl/index_read.cpp 修改： thirdparty/faiss/faiss/impl/index_write.cpp 修改： thirdparty/faiss/faiss/impl/pq4_fast_scan.cpp 修改： thirdparty/faiss/faiss/impl/pq4_fast_scan_search_qbs.cpp 修改： thirdparty/faiss/faiss/impl/simd_result_handlers.h 修改： thirdparty/faiss/faiss/index_io.h 修改： thirdparty/faiss/faiss/invlists/DirectMap.cpp 修改： thirdparty/faiss/faiss/invlists/DirectMap.h 修改： thirdparty/faiss/faiss/invlists/InvertedLists.cpp 修改： thirdparty/faiss/faiss/invlists/InvertedLists.h 修改： thirdparty/faiss/faiss/utils/binary_distances.cpp 修改： thirdparty/faiss/faiss/utils/bit_table.cpp 修改： thirdparty/faiss/faiss/utils/data_backup_file.cpp 修改： thirdparty/faiss/faiss/utils/data_backup_file.h 修改： thirdparty/faiss/faiss/utils/distances.cpp 修改： thirdparty/faiss/faiss/utils/distances.h 修改： thirdparty/faiss/faiss/utils/distances_if.h 修改： thirdparty/faiss/faiss/utils/partitioning_avx2.cpp 修改： thirdparty/faiss/faiss/utils/structure-inl.h 修改： thirdparty/faiss/tests/test_dealloc_invlists.cpp 修改： thirdparty/faiss/tests/test_distances_if.cpp 修改： thirdparty/faiss/tests/test_ivf_index.cpp 修改： thirdparty/faiss/tests/test_mem_leak.cpp 修改： thirdparty/hnswlib/examples/searchKnnCloserFirst_test.cpp 修改： thirdparty/hnswlib/examples/updates_test.cpp 修改： thirdparty/hnswlib/hnswlib/hnswlib.h 修改： thirdparty/hnswlib/hnswlib/space_hamming.h 修改： thirdparty/hnswlib/hnswlib/space_jaccard.h 修改： thirdparty/hnswlib/hnswlib/visited_list_pool.h 修改： thirdparty/hnswlib/main.cpp 修改： thirdparty/hnswlib/python_bindings/bindings.cpp 修改： thirdparty/hnswlib/sift_1b.cpp 修改： thirdparty/hnswlib/sift_test.cpp@liliu-z @Light-City
- About the clnag-format issue plz check the follow item. It reported error of
include/knowhere/dataset.h - About the UT issuem, plz check the follow item.
- I have already triggered the E2E CI Thanks
Hi @Light-City ,
there is a slight format issue
include/knowhere/dataset.h
====================
--- original
+++ formatted
@@ -42,7 +42,7 @@
std::is_same_v<T, const int64_t*>) {
delete[] arg;
} else if constexpr (std::is_same_v<T, const void*>) {
- delete[] (char*)(arg);
+ delete[](char*)(arg);
}
},
x.second);
@Light-City Plz check if the UT failure is caused by this PR. Since this PR is opened for a long time, it can be some other bugs get fixed, so if not sure about the cause of this UT failure, feel free to rebase this PR, and I can help rerun tests again