llama_index
llama_index copied to clipboard
GPTListIndex does not do well
Since my file was large GPT would timeout with GPTSimpleVectorIndex. So I split the document into multiple files and saved though GPTSimpleVectorIndex. After slitting I merged them using GPTListIndex. But the quality of results are remarkable. Querying he index created using GPTListIndex does not even appear to be contextual, its getting data from other source. Is there any other way to merge all the indices created by GPTSimpleVectorIndex into one?
document1 = loader.load_data('jason_nm-_MConverter.eu_-1-500.pdf')
document2 = loader.load_data('jason_nm-_MConverter.eu_-500-1000.pdf')
document3 = loader.load_data('jason_nm-_MConverter.eu_-1001-1500.pdf')
document4 = loader.load_data('jason_nm-_MConverter.eu_-1501-2000.pdf')
document5 = loader.load_data('jason_nm-_MConverter.eu_-2001-2621.pdf')
documentF = loader.load_data('nm.pdf')
index1 = GPTSimpleVectorIndex(document1)
index2 = GPTSimpleVectorIndex(document2)
index3 = GPTSimpleVectorIndex(document3)
index4 = GPTSimpleVectorIndex(document4)
index5 = GPTSimpleVectorIndex(document5)
index1.set_text("summary1")
index2.set_text("summary2")
index3.set_text("summary3")
index4.set_text("summary4")
index5.set_text("summary5")
index6 = GPTListIndex([index1, index2, index3, index4, index5])
response = index6.query('How to get out of addiction')
print(response)