marvin
marvin copied to clipboard
large memory footprint introduced by `chromadb` default embeddings
ChromaDB uses sentence-transformers
by default for embeddings, which requires torch
, which all together makes the footprint of chroma something like ~5GB.
We use OpenAI's text-embedding-ada-002
model offered via chromadb.utils.embedding_functions
, but
chromadb
enforces the sentence-transformers
dependency at this time.
We should find a way around this if chroma doesn't make sentence-transformers
an optional dep
#87 makes chromadb
an optional extra, but will keep this open as it seems chroma is likely to make sentence-transformer
optional
@zzstoatzz chroma 0.4.0
is much smaller now.