open-telemetry
Add Model Context Protocol (MCP) project proposal
Resolves https://github.com/open-telemetry/community/issues/3129
Related to https://github.com/open-telemetry/opentelemetry.io/pull/8331
Count me down for being a contributing member! Looking forward to this SIG. Here's a related thread in the #otel-semantic-conventions slack channel from a couple weeks ago that may be of interest to this SIG.
Thanks for the review @niwoerner . I have updated the proposal based on your feedback.
A few notes --
- I think the scope of this is very broad as it stands, touching multiple discrete components/parts of the project with divergent use cases and goals.
- I think there's discrete use cases (eg, mcp endpoints in Weaver, or ability to do config validation) that are valuable, but I also think that these should probably be accepted at a per-component level. We don't necessarily need a dedicated project for adding weaver support to weaver, or the collector, for example.
- I also think that there's a little bit of an argument for "hey, is anyone asking for this?" I think we do need to be able to experiment like this, but I also think that there's some level of... "hey, if people want to go off and build these MCP servers as an extension and then get a user base and try to upstream it" then that's fine. It's different to have stuff with proven fit/use cases.
I think it'd be totally appropriate to try and build a community around this independent of the main project and consider how different otel components could integrate MCP. I even think that there's smaller/point stuff (eg, a config validator MCP for the collector) that should be addressed at the existing SIG level.
Question based on the discussion we had at the GC call today: would the Docs SIG be a good initial place for this SIG?
Note: I'm referring to AI-Tooling instead of MCP-servers because I believe not every functionality related to AI would require a MCP-Server but still bring a benefit for the OTel ecosystem
There's a huge potential to simplify the adoption/implementation of OTel with AI-Tooling and as mentioned the scope of potential OTel components which could benefit from is broad.
While I agree that the concrete implementation should happen on component level in communication with the affected SIG, there are a few challenges which come with that.
We'll end up with different implementations + inconsistencies across those tools, resulting in different user/developer-experiences. Additionally there might be redundant efforts for tooling with similar goals if there is no coordination happening across SIGs. Also, It would be great for users to have a common place to look at "Which official AI-Tooling is available in context of OTel today? What is currently developed and might be available soon?".
A "MCP project/SIG" is perhaps not be the right term, but I believe what @pavolloffay and myself are looking for is a shared place to track and align the development of AI-Tooling in context of OTel.
Having a cross-cutting SIG could be the right place to coordinate the development of AI-Tooling as it'd impact several different implementation/specification SIGs.
I see the point that there is no critical need for a dedicated SIG to be able to experiment/develop this type of tools right now - so I understand the idea of placing this project into an existing SIG and based on demand eventually split it to a later point.
Could the Developer Experience SIG be a good starting point? At the end of the day, the goal of AI-Tooling is to facilitate the usage of OpenTelemetry.
We'll end up with different implementations + inconsistencies across those tools, resulting in different user/developer-experiences.
I fully support this view. It is vital that we coordinate to avoid redundancy, as overlapping tools will negatively impact the MCP's effectiveness.
Having a cross-cutting SIG could be the right place to coordinate the development of AI-Tooling as it'd impact several different implementation/specification SIGs.
Exactly. A key driver for this proposal is to establish a centralized forum for discussion, acknowledging that the actual implementation will be distributed across various components (such as the collector for config schema retrieval).
Could the Developer Experience SIG be a good starting point? At the end of the day, the goal of AI-Tooling is to facilitate the usage of OpenTelemetry.
It works for me, If we can promote the MCP topics in that SIG and other people can join.
Just to keep the issue up to date, @dmathieu, @pavolloffay and myself have discussed that in the DevEx SIG Meeting today and we will have another round of discussions next week. We did agree that the goals of the DevEx SIG and the MCP proposal is to improve the developer experience and user onboarding, so we may be able to integrate MCP into the DevEx SIG.
Our current main concern is about the work we have ongoing, but as there are discussions about moving the next steps to End User SIG with the OTel blueprints proposal, we may end this chapter of DevEx and MCP would be happily onboarded.
Also tagging @tsloughter for visibility.
I am a big fan of having this as a part of an existing SIG (DevEx) vs spinning up a dedicated one, it's a natural home to get this project kicked off, and if it grows bigger than having a dedicated SIG is still an option. We do a few other projects by that pattern already, and I start to like it that way more than adding yet another SIG everytime we kick off a proposal.
Overall, after having some time to think about this and talk with people about it, I think it's a good idea to have this.
Two major comments on the proposal itself:
- I think we have to make clear that "OpenTelemetry MCP" like any other building block of our project is concerned with the instrumentation, generation, collection and export of telemetry data. Everything on the right hand side which is concerned with "working with this data" is out of scope. This means that the MCP server should not include something like a layer to query data in a backend, etc. I understand that this is also an exciting topic, but similarly to the Telemetry Query Language project we rejected a while ago, I argue this is out of scope. Although I would love to see a OSS project emerging either standalone or as part of something existing (jaeger?) that covers that part.
- I am not sure how this works exactly with the MCP server, but the data that it consumed internally should be sourced from a common place and not be "exlusive" to the MCP, I suspect that there is some synergy with the ecosystem explorer project (#3000), cc @jaydeluca
To follow up on what @julianocosta89 said, the DevEx SIG met again yesterday and we are all in agreement on the MCP project being incorporated.
@svrnm @trask @jpkrohling @tsloughter @niwoerner the PR is updated.
The CI failure seems to be unrelated to this PR.
Call for contributors blog post (it's just a draft for now https://github.com/open-telemetry/opentelemetry.io/pull/8629 )