vault-ai icon indicating copy to clipboard operation
vault-ai copied to clipboard

How accurate is this?

Open Uday-461 opened this issue 2 years ago • 5 comments

How accurate will this be? Will it just do a search on the data and return the closest possible match? Or can I upload a large amount of data and have it analyse the data to give me inference from one possible document from the data?

Uday-461 avatar Apr 18 '23 10:04 Uday-461

Also is it compatible with GPT3.5 turbo, or GPT4, or both?

Uday-461 avatar Apr 18 '23 10:04 Uday-461

Your API key from OpenAI is probably for 3.5 Turbo, so...

DoingFedTime avatar Apr 18 '23 11:04 DoingFedTime

I compared response results. See issue here: https://github.com/pashpashpash/vault-ai/issues/26#issue-1676657944

gitardav avatar Apr 20 '23 15:04 gitardav

By default it only grabs 3 of what it thinks are the most related documents and tries to stuff contents into a single query.

Do not treat this as a summarizer. Do not assume that it takes into account/thinks-about or processes ALL of your documents.
Of potentially thousands of uploaded documents. It will grab 3 and stuff as much of the content from those three documents into the query as it can while fitting the 4097 token allowance. In some cases it will cut some of those documents out if they're too lengthy.

isConic avatar Apr 21 '23 01:04 isConic

Worth Noting. It will grab 3 documents based on wether or not the documents, when summarized via embeddings, are semantically similar to your query.

isConic avatar Apr 21 '23 01:04 isConic