llama_index icon indicating copy to clipboard operation
llama_index copied to clipboard

GPTListIndex does not do well

Open azmathr opened this issue 1 year ago • 0 comments

Since my file was large GPT would timeout with GPTSimpleVectorIndex. So I split the document into multiple files and saved though GPTSimpleVectorIndex. After slitting I merged them using GPTListIndex. But the quality of results are remarkable. Querying he index created using GPTListIndex does not even appear to be contextual, its getting data from other source. Is there any other way to merge all the indices created by GPTSimpleVectorIndex into one?

document1 = loader.load_data('jason_nm-_MConverter.eu_-1-500.pdf')
document2 = loader.load_data('jason_nm-_MConverter.eu_-500-1000.pdf')
document3 = loader.load_data('jason_nm-_MConverter.eu_-1001-1500.pdf')
document4 = loader.load_data('jason_nm-_MConverter.eu_-1501-2000.pdf')
document5 = loader.load_data('jason_nm-_MConverter.eu_-2001-2621.pdf')
documentF = loader.load_data('nm.pdf')

index1 = GPTSimpleVectorIndex(document1)
index2 = GPTSimpleVectorIndex(document2)
index3 = GPTSimpleVectorIndex(document3)
index4 = GPTSimpleVectorIndex(document4)
index5 = GPTSimpleVectorIndex(document5)

index1.set_text("summary1")
index2.set_text("summary2")
index3.set_text("summary3")
index4.set_text("summary4")
index5.set_text("summary5")

index6 = GPTListIndex([index1, index2, index3, index4, index5])

response = index6.query('How to get out of addiction')
print(response)

azmathr avatar Mar 13 '23 07:03 azmathr