vearch
vearch copied to clipboard
Distributed vector search for AI-native applications
Overview
Vearch is a scalable distributed system for efficient similarity search of deep learning vectors.
Architecture
-
Data Model
space, documents, vectors, scalars
-
Components
Master
,Router
andPartitionServer
-
Master
Responsible for schema mananagement, cluster-level metadata, and resource coordination.
-
Router
Provides RESTful API:
create
,delete
search
andupdate
; request routing, and result merging. -
PartitionServer (PS)
Hosts document partitions with raft-based replication.
Gamma is the core vector search engine implemented based on faiss. It provides the ability of storing, indexing and retrieving the vectors and scalars.
Quick start
-
One-click binary installation, please see docs/BinaryInstallation.md(中文版).
-
Quickly compile the source codes to build a distributed vector search system with RESTful API, please see docs/SourceCompileDeployment.md.
-
Vearch can be leveraged to build a complete visual search system to index billions of images. The image retrieval plugin for object detection and feature extraction is also required. For more information, please refer to docs/Quickstart.md.
-
Vearch Python SDK enables vearch to use locally. Vearch python sdk can be installed easily by pip install vearch. For more information, please refer to docs/APIPythonSDK.md.
APIs and Use Cases
LowLevelAPI
- docs/APILowLevel.md
- For GPU docs/APILowLevel.md
VisualSearchAPI
- docs/APIVisualSearch.md
PythonSDKAPI
- docs/APIPythonSDK.md
Document
- https://vearch.readthedocs.io/en/latest
- https://vearch.readthedocs.io/zh_CN/latest
- Common QA
Benchmarks
- benchmarks
Publication
Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu Chen, Zhenyun Ni, Ning Wang, Yuan Chen. The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform. In the 19th International ACM Middleware Conference, December 10–14, 2018, Rennes, France. https://arxiv.org/abs/1908.07389
Community
You can report bugs or ask questions in the issues page of the repository.
For public discussion of Vearch or for questions, you can also send email to [email protected].
Our slack : https://vearchwrokspace.slack.com
Known Users
Welcome to register the company name in this issue: https://github.com/vearch/vearch/issues/230 (in order of registration)
欢迎在此 issue https://github.com/vearch/vearch/issues/230 中登记公司名称
Wechat official communication group
License
Licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.