Mintplex-Labs
[FEAT]: Support Model Context Protocol (MCP)
What would you like to see?
Supporting MCP https://github.com/modelcontextprotocol turns anthingllm into a powerful client.
There is little need for us to add MCP directly when you can use the TS/JS client to generate new Agent Skills and import those into AnythingLLM and then have the benefit of our tool calling but enabled to build your tools in a framework you know.
https://docs.anythingllm.com/agent/custom/developer-guide
The over 700 existing MCP servers is a decent amount of reason.
OK, that is great - I agree! Nobody mentioning MCP has helped define what kind of flow they would like to see in the app. Its an extremely broad ask with no clear end goal for us to achieve.
I am not against MCP in any way, but "support MCP" gives me no indication of the following:
- How people are currently using it (tools, workflows, etc)
- Expectations on how that should work for them wrt AnythingLLM
The only straightforward ask I can even predict from this is having it function like Claude Desktop? Where there is basically a collection of MCPs that start on boot and are defined to be loaded as tools, but I'm making an assumption about the use case here and we can never support python MCPs, which i imagine 90% of those 700 are.
The issue isn't closed because it's a wontfix, the issue is poorly described and has no definitive action for a feature request - that is why.
I am using MCPs in both Claude Desktop, and Cline, and looking for ways to expand usage.
Web search, file access, markdown parsing, web scraping are some of the mcps I've used to do work.
Because Claude desktop app supports them so well, that's where I'm spending almost all my time these days. I've tried to get other clients with mcp support working and haven't yet. That being said, I'm starting to use Cline for more tasks now that they have the plan mode.
For me, MCP's have changed the baseline of capabilities.
@tranqy And for your MCPs, are they written in Python or TS? For importing of MCPs, how would that look for you wrt AnythingLLM? Like just dragging the folder with its server code into AnythingLLM and now that is an available skill?
These are the details we wanted to really know. We arent going to rip out our built-in framework for a 3rd party flow simply because of the lack of flexibility with other systems that are needed, but I would be happy to have MCPs work via a compatibility layer so your work is not duplicated.
The protocol, as I understand it, is the client application will get a command line to run, and then communicate with the MCP over stin/stout. The way I configure them with Cline and Claude is to drop in a JSON block into a larger JSON file, so it's def a more technical oriented workflow currently. There are apps that automate the modification of the JSON file (see smithery), I don't use them.
It's up to the end user to ensure the framework is setup before setting up the MCP. Most MCPs init with npx or uv, but there are others as well. From the AnythingLLM perspective, I think you string contact, exec, and if it fails tell the user to check the command.
Once server is open, then calls need to be marshalled to the service via stin/stout. Here is where my understanding gets very light. When I'm using them, you can see when the AI is going to delegate to the MCP, it often asks for permission, then shifts the (I suspect) command to stin.
From my perspective it would be awesome if AnythingLLM supported it, I was using AnythingLLM more before MCPs, and would love to have options outside the Claude official app / Cline for effective MCP usage, but I understand there is significant work already in other options.
@tranqy Thank you for being so descriptive here - okay reopened task since now I get what is being asked :)
I'd be happy to help if any questions come up, I'm a heavy MCP user. I also just stumbled on vectoradmin at cruising altitude, plan to install that in the next couple days, thank you and the team for your work!
@tranqy And for your MCPs, are they written in Python or TS?
There is MCP TypeScript SDK https://github.com/modelcontextprotocol/typescript-sdk and a lot ts MCP Servers.
I'm particularly interested in MCP Stagehand . I think it's the right way to support OS Agent (computer use) in AnythingLLM.
Re: @tranqy's description of MCP and @therealtimex's note about TS support: I agree that supporting MCP in AnythingLLM would be super useful. But my understanding is that MCP servers can't run directly in a browser environment due to security sandboxing (Anthropic currently only supports MCP in their desktop app). Both TypeScript and Python MCP servers operate as standalone processes requiring system-level access (stdio streams, child processes, filesystem) that would be blocked in a browser environment.
I did some research in case helpful—though I should preface this by noting I haven't worked with these solutions, so this is just my understanding and there may be some technical nuances I'm not familiar with. That said, my understanding is that you can go two ways to support MCP:
- Local Bridge (like Claude Desktop):
- Run a backend service that spawns/manages MCP servers and has bidirectional stdio communication
- Expose an HTTP/WebSocket API to the browser
- Server-Sent Events (SSE):
- Run MCP servers as web services (local or remote), and allow browsers to connect via SSE streams
- This is supported by MCP SDKs
- LibreChat notes this works better for multi-user scenarios
- Supporting SSE is relatively straightforward (if the MCP server supports SSE, users just need to run it via terminal command and AnythingLLM can connect to its SSE endpoint)
- However, not all MCP servers support SSE out of the box so you need to implement a local bridge if you want to support stdio-only MCP servers (or have the MCP servers run automatically when AnythingLLM runs)
There are some existing solutions that would be worth checking out:
- LibreChat supports both approaches
- There are libraries like mcp-proxy (Glama.ai), MCP-Bridge (EvalsOne), and Remote-MCP (ssut) that launch stdio-only MCP servers and expose an SSE endpoint the browser can access
This project could be useful for ideas of how to implement. It can interact with MCP servers. https://github.com/modelcontextprotocol/inspector
Anything LLM needs to be a MCP client. Not a server. Right?
@goughjo02 We would likely have to also be the server as well - since then it would depend on the user to start these servers which most wont know how to do.
@15626471095 - we hope EOM April.
MCP is adding HTTP and authentication support very soon (changes are merged, waiting for release) which will allow third parties to deploy their remote MCP servers and MCP clients to connect to them via HTTP and with authentication, therefore removing the need of hosting the server locally. I would say we can expect new MCP version very soon.
More on that here (Approved section): https://github.com/orgs/modelcontextprotocol/projects/2
I believe this might influence the implementation on AnythingLLM side in the foreseeable future as remote MCP servers from third parties will start popping up as soon as new MCP version is out (discoverability still wont be but is in their roadmap as well).
Love what you are doing with AnythingLLM, keep it up!
I expect, though a lot of tools probably will stick with the basic implementation. That being said, connecting to an external server endpoint to load in tools and delegate should be pretty trivial to support. As well as multiple connections to multiple MCP servers hosting an arbitrary number of tools.