agnaistic
Feature request: Support a vector database
Supporting a vector database like ChromaDB would have numerous benefits including a longer memory.
There are plans to start bringing in features like this. If you have any expertise with ChromaDB, your input or what the implementation would like would greatly be appreciated over in the Discord or here.
Unfortunately I don't however there has been some (unfinished) work on this recently in Tavern that might be a helpful reference: https://github.com/Cohee1207/SillyTavern/pull/333 https://github.com/Cohee1207/SillyTavern-extras/issues/23 and oobabooga implementation: https://github.com/oobabooga/text-generation-webui/tree/main/extensions/superbooga
I used weaviate:
- https://github.com/weaviate/weaviate?utm_source=google&utm_medium=cpc&utm_campaign=17432327206&utm_content=142705996452&utm_term=weaviate&gclid=CjwKCAjwpayjBhAnEiwA-7ena-EGwc_xzO26FW7fjN8R2hcIMxXouwpeIvQSKWonxtRk-5t9XUEB-RoCs5AQAvD_BwE And Pinecone (free account):
- https://www.pinecone.io/
Chromadb seems to be focussed towards certain LLMs and is embeddings.
I think ChromaDB will give the best results if used correct but will be the hardest to use in a 'one size fits all' scenario, since it runs a selected LLM index
Those two look promising, however pinecone appears to be a PaaS and weaviate needs to be installed. Weaviate definitely looks promising, but I need a low barrier for entry which ChromaDB satisfies by being installable via pip/poetry.
Weaviate client is installed with pip install weaviate-client or even better npm i weaviate-ts-client
Keep in mind that the server part in all cases uses very large amounts of RAM/VRAM as in A100-80GB (recommended in this list https://www.e2enetworks.com/blog/top-7-vector-databases-for-ai). I am running weaviate and it takes up 32GB of memory on a dev server, this is why I recommended Pinecone because of the large free tier and most automation gpt packages (autogpt,babyagi) support it for that reason.
Another vector db to not overlook is Milvus
I tried all of the above and weaviate for me was quickest to implement due to a docker for the server and npm package for the client, zero config.