nodejs-firestore icon indicating copy to clipboard operation
nodejs-firestore copied to clipboard

Add similarity threshold to (.findNearest) queries in addition to returning similarity scores.

Open Moe03 opened this issue 1 year ago • 6 comments

Is your feature request related to a problem? Please describe. I'm trying to be able to add a thresold to my similarity search with vectors but it doesn't semm to be mentioned here anywhere? https://firebase.google.com/docs/firestore/vector-search#node.js_2

Describe the solution you'd like Adding a field called similarityThreshold to the vector search query would be perfect:

const searchRes = await dbToUse.collection(`voiceglow/${agentData.ID}/kb_chunks`).findNearest("vector", 
   FieldValue.vector(searchVector), {
   limit: 5,
   distanceMeasure: 'COSINE',
   similarityThresohold: 0.2 // add this (currently doesn't work)
}).get();

Also adding to the queryies of such types (.findNearest) the similarity score with the search vector is highly beneficial for debugging purposes, we're currenlty using pgvector on postgres but would love to migrate everything on firestore but these may be annoying to deal with.

Describe alternatives you've considered There is nothing mentioned in the docs, I hope it's already implemented but not docuemented/typed properly?

Moe03 avatar Jun 30 '24 22:06 Moe03

@MarkDuckworth Could you take this into consideration as part of your API design?

tom-andersen avatar Jul 02 '24 14:07 tom-andersen

+1 this would be great to have

dave-lacy avatar Jul 03 '24 17:07 dave-lacy

Thanks for the feedback. I've passed it along to the rest of the team working on vector search in Firestore.

I can confirm that similarity threshold and returning the vector distance is not supported as of today. The Firestore vector search feature is currently in Preview and it is not necessarily feature complete.

MarkDuckworth avatar Jul 08 '24 16:07 MarkDuckworth

+1 definitely needed

joshke avatar Aug 29 '24 13:08 joshke

This is supported in Javascript and other languages, how can we speed up the process? would a PR be accepted?

manucorporat avatar Oct 01 '24 06:10 manucorporat

Hello!

Check this https://github.com/googleapis/nodejs-firestore/issues/2217. It might help.

pogorv12 avatar Oct 15 '24 06:10 pogorv12