dspy icon indicating copy to clipboard operation
dspy copied to clipboard

Added support for variable sized output vectors via Nomic Embed v1.5

Open AndriyMulyar opened this issue 4 months ago • 3 comments

Added support for Nomic Embed v1.5: https://blog.nomic.ai/posts/nomic-embed-matryoshka

This let's you specify a vectorizer that will generate much smaller but still performant embeddings for local use.

AndriyMulyar avatar Feb 15 '24 23:02 AndriyMulyar

Wonderful, thanks a lot. Merging after checks.

okhat avatar Feb 17 '24 05:02 okhat

Ok we have a small problem:

import nomic

I guess normally we deal with this by importing optional dependencies either:

  1. Inside the class
  2. Outside the class, but in try/except form and on except just set it to None or something

okhat avatar Feb 17 '24 05:02 okhat

Will update.

AndriyMulyar avatar Feb 17 '24 23:02 AndriyMulyar

Hi @AndriyMulyar , just following up on this update. looks good to merge after editing the import. Feel free to use examples from other retrievers on how to optionally import the local dependencies

arnavsinghvi11 avatar Mar 24 '24 20:03 arnavsinghvi11