Support for model router pattern for chains to allow for dynamic routing to right chains based on vector store semantics

[keshy]

We need a model gateway pattern support for Chains for following reasons:

• We may have external decision making elements that would help route which LLM model would need to handle a request. E.g. vector store.
• Agents don't always cut it with the customization that'd be needed. We will need custom tool building etc which is overkill for a simple routing use case.
• Downstream LLMs can also be any of the numerous chain types that can be supported. This allows for a scalable LLM orchestration model with chaining beyond what's supported today which is mainly Sequential chains.

[lifan0127]

I agree model routing is a better terminology than agent/tool for such use cases.

[Medeirox]

Does anyone knows why the chains.router package in Langchain isn't available? I had to download the source code and enable it by myself. The documentation talks about it in a way that it seems to be available.

https://python.langchain.com/en/latest/modules/chains/generic/router.html?highlight=router

[dosubot[bot]]

Hi, @keshy! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is requesting support for a model router pattern for chains in order to enable dynamic routing based on vector store semantics. There has been agreement from lifan0127 that "model routing" is a better terminology for such use cases. Additionally, Medeirox has raised a question about the availability of the chains.router package in LangChain and has provided a link to the documentation for reference.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project!