Best llm to use gpt_index

[Keith-Hon]

OpenAI's models definitely have high quality. But suppose someone wants to achieve comparable results using self host models and gpt-index, what are the alternatives?

[AldawsariNLP]

see this issue https://github.com/jerryjliu/gpt_index/issues/423#issue-1579871856

[Keith-Hon]

see this issue #423 (comment)

Yeah I have looked at it. While we could use custom models with some modifications on our own, it seems to me that most open source models didn't get trained with external data sources in mind. Which could not leverage most of the power from library like gpt-index or langchain.

[logan-markewich]

Another concern is input length. Most open-source models have a smaller max length (OPT is 2048, FLAN-T5 is 512!!).

I think in the next few months though, we will see more open-source models with capabilities similar to OpenAI models. The research area is pretty hot right now.

Existing open-source models do not perform well with gpt_index, at least the ones I've tried (GPT-J, OPT, FLAN-T5). I think FLAN-T5 works the best (xl and xxl especially), I just wish it accepted longer input sequences.

[jerryjliu]

Since this is more of an issue of choosing the LLM and not an issue of how to use it with GPT Index (i think), i'm going to close the issue for now.

Btw even if langchain doesn't support the llm you're looking to integrate, you can always subclass LLMPredictor to integrate your LLM. ping me if you get to that and i can help