paper-qa icon indicating copy to clipboard operation
paper-qa copied to clipboard

wrong citations if initiate the Docs object with texts_index

Open XiaomeiLi1 opened this issue 2 years ago • 2 comments

Looking at how to convert the langchain Document object to the paper-qa Docs object. Currently, my solution is below.

splitter = splitter_type(chunk_size=chunk_size, chunk_overlap=chunk_overlap)
docs_split = splitter.split_documents(docs) # docs is langchain Document object

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key) vectorstore = FAISS.from_documents(docs_split, embeddings, ids=[doc.metadata["source"] for doc in docs_split]) docs_obj = Docs(texts_index= vectorstore, memory=True) response = docs_obj.query(question)

Then I found it cannot pick up (it makes up) the citations. Do you have better solutions for this? BTW, I don't have PDF files downloaded. They are just stored in the langchain Document object. Thanks a lot!

XiaomeiLi1 avatar Nov 14 '23 03:11 XiaomeiLi1

Strange - do you have some example input/output? Hard to tell what could be going wrong.

whitead avatar Nov 17 '23 18:11 whitead

Any guidance how to integrate with external vector db such as chromadb? I don't find any documentation for the same.

nandhaece07 avatar Dec 27 '23 08:12 nandhaece07

Any guidance how to integrate with external vector db such as chromadb? I don't find any documentation for the same.

This was asked for in https://github.com/Future-House/paper-qa/issues/153, you can see some information there.

Also, we have just released version 5, which more or less rewrites everything. If you are still stuck, please reopen a new issue using paper-qa>=5

jamesbraza avatar Sep 11 '24 18:09 jamesbraza