mongoengine icon indicating copy to clipboard operation
mongoengine copied to clipboard

using vector search with mongoengine

Open alexisdal opened this issue 2 years ago • 2 comments

hello

I have been using mongoengine on my latest project and I'm happy with it.

somehow, I would like to push vectors (embeddings) into my mongodb collection and perform vector searches. It's apparently a fresh development of mongodb. source => https://www.mongodb.com/products/platform/atlas-vector-search

I made a simple lookup in the documentation and issues of mongoengine but did not find any matches.

what I want to achieve

  1. push vectors in a collection
  2. perform similarity search between a vector and the entire collection, retrieve the K-th closest matches (typically using cosine distance)
  3. using the mongoengine abstraction layer

Any clue about how to do this ?

alexisdal avatar Dec 12 '23 09:12 alexisdal

There is no "specific" support built in MongoEngine but I'm not sure it's worth having one. There is an API for creating indexes in pymongo raw format, and an api for running aggregation pipeline also in raw format, have you tried to make use of these?

bagerard avatar Dec 27 '23 20:12 bagerard

From what I know about the vector search is part of the elastic search solution that Atlas provide. Maybe this library can be helpful https://github.com/certego/AtlasQ

idoshr avatar Jan 05 '24 14:01 idoshr