mcp-context-forge [Epic]: Implement SEP-1649 MCP Server Cards Discovery

[Epic]: Implement SEP-1649 MCP Server Cards Discovery

Open jonpspri opened this issue 1 month ago • 0 comments

🧭 Type of Feature

Please select the most appropriate category:

[x] Enhancement to existing functionality
[x] New feature or capability
[ ] New MCP-compliant server
[ ] New component or integration
[ ] Developer tooling or test improvement
[ ] Packaging, automation and deployment (ex: pypi, docker, quay.io, kubernetes, terraform)
[ ] Other (please describe below)

🧭 Epic

Title: MCP Server Cards Discovery via .well-known URI (SEP-1649)

Upstream Reference: SEP-1649: MCP Server Cards

Goal: Enable AI agents and MCP clients to discover MCP Gateway capabilities, available tools, resources, and prompts through standardized .well-known/mcp.json endpoints without requiring connection establishment or manual configuration. This implements the upstream MCP specification enhancement proposal SEP-1649 for HTTP-based server discovery.

Why now:

The upstream MCP community is actively developing SEP-1649 as a standardized discovery mechanism
AI service discovery and execution without manual reconfiguration is a critical use case for production deployments
Implementing this now positions the gateway as an early adopter and reference implementation
Enterprise users need automated discovery for dynamic service composition and registry crawling
Reduces integration friction by eliminating manual endpoint configuration

🙋♂️ User Story 1: AI Agent Dynamic Discovery

As an: AI agent (Claude, GPT, Gemini, etc.) I want: To discover available MCP services by fetching /.well-known/mcp.json from the gateway So that: I can automatically learn what tools, resources, and prompts are available without requiring internal reconfiguration or manual setup

✅ Acceptance Criteria

Scenario: AI agent discovers gateway capabilities
  Given an MCP Gateway is deployed at https://gateway.example.com
  When the AI agent performs GET /.well-known/mcp.json
  Then it receives a JSON response with protocol version, server info, and capabilities
  And the response includes transport configuration (type: "streamable-http", endpoint: "/mcp")
  And the response lists available tools (either static array or "dynamic" marker)
  And the response lists available resources (either static array or "dynamic" marker)
  And the response lists available prompts (either static array or "dynamic" marker)

Scenario: AI agent uses discovered endpoint
  Given the AI agent has fetched the server card from /.well-known/mcp.json
  When the AI agent extracts the transport endpoint ("/mcp")
  Then it can make JSON-RPC requests to POST /mcp without additional configuration
  And it can invoke tools using the discovered tool names
  And it can read resources using the discovered resource URIs

🙋♂️ User Story 2: Gateway Administrator Configuration

As a: Gateway administrator I want: The gateway to automatically generate and serve MCP server cards reflecting current configuration So that: Clients can always discover up-to-date capabilities without manual documentation updates

✅ Acceptance Criteria

Scenario: Server card reflects current gateway state
  Given the gateway has 5 active virtual servers with 25 total tools
  When I query GET /.well-known/mcp.json
  Then the server card includes the current protocol version (2025-06-18)
  And the server card shows accurate capability flags (tools.listChanged: true)
  And the server card indicates dynamic tool discovery is required

Scenario: Server card respects authentication configuration
  Given the gateway has AUTH_REQUIRED=true
  When I query GET /.well-known/mcp.json
  Then the server card includes authentication.required: true
  And the server card lists supported authentication schemes (bearer, oauth2)

🙋♂️ User Story 3: Registry Crawler Integration

As a: MCP registry maintainer or service catalog I want: To crawl multiple gateway domains and automatically index available services So that: I can build comprehensive service directories without manual registration processes

✅ Acceptance Criteria

Scenario: Registry crawls gateway domain
  Given a list of gateway domains to index
  When the registry crawler fetches /.well-known/mcp.json from each domain
  Then it receives standardized server cards from compliant gateways
  And it can extract server name, version, and description
  And it can catalog available tools, resources, and prompts
  And it can store transport endpoints for client configuration

Scenario: Registry detects gateway updates
  Given a gateway has been previously indexed
  When the gateway adds new tools or updates capabilities
  And the registry re-fetches /.well-known/mcp.json
  Then it detects the changes via version or timestamp comparison
  And it updates its index with the new capabilities

🙋♂️ User Story 4: MCP Resource-Based Discovery

As an: MCP client with an established connection I want: To access the server card as an MCP resource at mcp://server-card.json So that: I can discover server metadata through the standard MCP protocol without requiring HTTP access

✅ Acceptance Criteria

Scenario: Client reads server card via MCP resource
  Given an MCP client has initialized a connection to the gateway
  When the client calls resources/read with URI "mcp://server-card.json"
  Then it receives the same server card content as /.well-known/mcp.json
  And the resource MIME type is "application/json"
  And the resource contains all capability and transport information

Scenario: Server card resource is always available
  Given the gateway supports the MCP resource protocol
  When the client lists available resources via resources/list
  Then "mcp://server-card.json" appears in the resource list
  And it is marked as a static resource (not dynamic)

📐 Design Sketch

sequenceDiagram
    participant AI as AI Agent
    participant GW as MCP Gateway
    participant VS as Virtual Servers
    participant MCP as MCP Servers

    Note over AI,GW: Discovery Phase (No Auth Required)
    AI->>GW: GET /.well-known/mcp.json
    GW->>VS: Query active virtual servers
    GW->>GW: Generate server card
    GW-->>AI: Server Card JSON<br/>(protocol, capabilities, endpoint)

    Note over AI,GW: Connection Phase
    AI->>GW: POST /mcp (initialize)
    GW-->>AI: InitializeResult

    Note over AI,GW: Tool Discovery Phase
    AI->>GW: POST /mcp (tools/list)
    GW->>VS: Aggregate tools from virtual servers
    VS->>MCP: Fan-out to peer MCP servers
    MCP-->>VS: Tool definitions
    VS-->>GW: Aggregated tools
    GW-->>AI: Complete tool list

    Note over AI,GW: Tool Invocation Phase
    AI->>GW: POST /mcp (tools/call: "tool_name")
    GW->>VS: Route to appropriate virtual server
    VS->>MCP: Forward to backend MCP server
    MCP-->>VS: Tool result
    VS-->>GW: Result
    GW-->>AI: Tool execution result

📐 Server Card Schema

The server card follows SEP-1649 specification:

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/mcp-server-card/v1.json",
  "version": "1.0",
  "protocolVersion": "2025-06-18",
  "serverInfo": {
    "name": "mcp-context-forge",
    "title": "MCP Context Forge Gateway",
    "version": "0.8.0"
  },
  "description": "Production-grade MCP gateway providing unified discovery, auth, rate-limiting, and federation",
  "iconUrl": "https://gateway.example.com/static/icon.png",
  "documentationUrl": "https://ibm.github.io/mcp-context-forge/",
  "transport": {
    "type": "streamable-http",
    "endpoint": "/mcp"
  },
  "capabilities": {
    "tools": {
      "listChanged": true
    },
    "resources": {
      "subscribe": true,
      "listChanged": true
    },
    "prompts": {
      "listChanged": true
    },
    "logging": {}
  },
  "authentication": {
    "required": true,
    "schemes": ["bearer", "oauth2"]
  },
  "instructions": "MCP Gateway federating multiple MCP servers. Use tools/list to discover available tools across all virtual servers.",
  "tools": ["dynamic"],
  "resources": ["dynamic"],
  "prompts": ["dynamic"]
}

🔗 MCP Standards Check

[x] Change adheres to current MCP specifications
[x] No breaking changes to existing MCP-compliant integrations
[ ] If deviations exist, please describe them below:

Compliance Notes:

Implements upstream SEP-1649 (currently in draft status in modelcontextprotocol/specification#1649)
Server card format follows proposed schema exactly
.well-known/mcp.json is the standardized discovery endpoint
mcp://server-card.json resource provides protocol-native access
Fully backward compatible - existing clients unaffected

Standards Alignment:

RFC 8615: Well-Known URIs (https://tools.ietf.org/html/rfc8615)
MCP Protocol 2025-06-18 specification
JSON Schema for validation

🔄 Alternatives Considered

Manual Configuration Only
- Rejected: Requires out-of-band communication and doesn't scale
- Forces users to manually document and distribute endpoint information
- Doesn't support dynamic discovery or registry crawling
Custom Discovery Endpoint (non-standard)
- Rejected: Creates vendor lock-in and reduces interoperability
- Example: /api/discover or /gateway/info
- Upstream community converging on .well-known/mcp.json standard
OpenAPI/Swagger for Discovery
- Rejected: MCP protocol semantics don't map cleanly to REST/OpenAPI
- Tools are invoked via JSON-RPC, not REST endpoints
- Server card provides MCP-specific metadata (capabilities, instructions)
DNS-SD / mDNS / Zeroconf
- Partially implemented: Gateway already supports mDNS discovery for local networks
- Limitation: Doesn't work across internet boundaries or provide detailed capability info
- Complementary: Can be used alongside .well-known for local auto-discovery
GraphQL Introspection
- Rejected: Introduces additional protocol complexity
- MCP already has tools/list, resources/list, prompts/list
- Server card provides initialization-time metadata without full connection

📓 Additional Context

Upstream Specification:

SEP-1649: MCP Server Cards: HTTP Server Discovery via .well-known
Status: Draft, actively discussed by MCP core maintainers
Authors: @dsp-ant, @nickcoai
Full Specification: See SEP-1649 Issue for complete technical details

Related Existing Implementation:

Gateway already implements /.well-known/mcp endpoint (routers/well_known.py)
Current implementation returns basic metadata but not full server card schema
Need to extend to full SEP-1649 compliance

Use Cases:

AI Agent Auto-Configuration: Claude Desktop, Cursor, Continue can auto-discover gateway
Service Registry: Build searchable catalogs of MCP services
Federation Discovery: Peer gateways can discover each other's capabilities
Documentation Generation: Auto-generate API docs from live server cards
Client Libraries: Enable SDK auto-configuration from gateway URL

Implementation Scope:

Update routers/well_known.py to generate SEP-1649 compliant server cards
Add server card generation logic using gateway's current configuration
Expose mcp://server-card.json as an MCP resource
Add configuration flags for customizing server card metadata (title, description, iconUrl)
Update documentation with examples of AI agent discovery workflows
Add integration tests for server card validation against JSON schema

Configuration Variables (new):

# Server Card Metadata
MCPGATEWAY_SERVER_TITLE="MCP Context Forge Gateway"
MCPGATEWAY_SERVER_DESCRIPTION="Production-grade MCP gateway"
MCPGATEWAY_SERVER_ICON_URL="https://gateway.example.com/icon.png"
MCPGATEWAY_SERVER_DOCS_URL="https://ibm.github.io/mcp-context-forge/"

# Discovery Features
MCPGATEWAY_DISCOVERY_ENABLED=true
MCPGATEWAY_DISCOVERY_INCLUDE_STATIC_TOOLS=false  # "dynamic" vs static array

Testing Strategy:

Unit tests: Server card generation from gateway state
Integration tests: HTTP GET /.well-known/mcp.json returns valid schema
E2E tests: AI agent discovers and invokes tools using only the gateway URL
Schema validation: JSON Schema validation of generated server cards
Compatibility tests: Verify existing clients unaffected

Documentation Updates:

docs/docs/architecture/adr/017-sep-1649-server-discovery.md (new ADR)
docs/docs/using/discovery.md (new guide)
docs/docs/api/well-known.md (update existing)
Update CHANGELOG.md with new feature

Related Issues:

Upstream: modelcontextprotocol/specification#1649
Related: .well-known URI handler (ADR-015)
Related: Federation discovery (ADR-008)

Oct 20 '25 09:10 jonpspri

mcp-context-forge mcp-context-forge copied to clipboard

[Epic]: Implement SEP-1649 MCP Server Cards Discovery

🧭 Type of Feature

🧭 Epic

🙋♂️ User Story 1: AI Agent Dynamic Discovery

✅ Acceptance Criteria

🙋♂️ User Story 2: Gateway Administrator Configuration

✅ Acceptance Criteria

🙋♂️ User Story 3: Registry Crawler Integration

✅ Acceptance Criteria

🙋♂️ User Story 4: MCP Resource-Based Discovery

✅ Acceptance Criteria

📐 Design Sketch

📐 Server Card Schema

🔗 MCP Standards Check

🔄 Alternatives Considered

📓 Additional Context

mcp-context-forge
mcp-context-forge copied to clipboard