huggingface
Can I hook it up to a retrieval system for a document chatbot?
I want to use the instructor-xl text embedding model and use FAISS to create and retrieve from a vector store. Sort of a chatbot for documents or a domain specific chatbot. Any ideas on how I can do it?
I would like to do the same, I'll try to mess around with it over the next week or two and see if I can't figure something out. Hopefully someone more knowledgeable can beat me to it though.
I am working on the same thing. Currently when I make the similarity search FAISS will always return the same vectors. I am using the same setup I have used for other embedding models. I guess this has something to do with the input format of encode() function. We give a list ["instruction", "text"]. Similarly when we do search on FAISS db it returns 2 arrays like this as distances:"[[0.47456202 0.49658185 0.5015992 ] [0.398577 0.4358151 0.4369905 ]]" (For k=3).
Any progress on this? I'm also interested in hooking up retrieval to the UI
Hi there, a was able to get this working. Actually I ended up using another embedding model (BAAI/bge-large-en) but instructor-xl was working fine too.
Initalize the model:
model = INSTRUCTOR('hkunlp/instructor-xl')
To get the embeddings of a query:
embedding = model.encode([["Represent the question for retrieving supporting documents: ", "Your query here"]], normalize_embeddings=True)
Then making similarity search on FAISS index:
//first load your index
index = faiss.read_index('path/to/your/index_file')
//make the search
D, I = index.search(embedding.astype('float32'), 3)
Hi there, a was able to get this working. Actually I ended up using another embedding model (BAAI/bge-large-en) but instructor-xl was working fine too.
Initalize the model:
model = INSTRUCTOR('hkunlp/instructor-xl')To get the embeddings of a query:
embedding = model.encode([["Represent the question for retrieving supporting documents: ", "Your query here"]], normalize_embeddings=True)Then making similarity search on FAISS index:
//first load your index index = faiss.read_index('path/to/your/index_file') //make the search D, I = index.search(embedding.astype('float32'), 3)
Where did you make these changes?
I'm also looking to implement this. Is there a place where a python script can take an input, be processed by Retrieval system, then send over output to model, as usual? I'm thinking maybe setting up a REST API for the python script and have the src/lib/server/generateFromDefaultEndpoint.ts file call that endpoint, so that the "message" includes the retrieved chunks as well.
Has anybody designed a chat agent and retrieval system from vector store db's like FAISS, ChromaDB?
