[CHORE] Proposal: Split Monorepo into Separate Repositories

Open crivetimihai opened this issue 1 month ago • 1 comments

Monorepo Split Proposal

Summary

Propose splitting the current MCP Gateway monorepo (~240K lines) into 14 focused repositories under a new GitHub organization github.com/contextforge-org/ to enable:

Independent versioning per component
Parallel development across teams
Multi-format packaging (PyPI, containers, Helm, binaries)
Clear ownership and focused repositories
Easier contribution and discoverability

Current State

Monorepo Size: ~240,000 lines of code

Core gateway: ~150K lines (FastAPI, 33 services, 11 routers)
Plugins: 40 Python plugins + 1 Rust plugin
MCP Servers: 4 Python + 5 Go servers
Infrastructure: Helm charts, Terraform, Ansible
Documentation: 172 markdown files
Tests: 288 test files

Pain Points:

Plugins tied to core release cycle
Large repository hard to navigate
Single CI/CD pipeline for everything
Conflicting versioning needs
Difficult to scale team ownership

Proposed Structure

graph TB
    subgraph ORG["GitHub Organization: contextforge-org"]
        subgraph CORE["Core Application"]
            GATEWAY["mcp-contextforge-gateway-core<br/>🐍 FastAPI Gateway<br/>📦 PyPI: mcp-contextforge-gateway<br/>🐳 Container: mcp-gateway"]
            UI["mcp-contextforge-gateway-ui<br/>🐍 HTMX + Alpine.js<br/>📦 PyPI: mcp-contextforge-gateway-ui"]
        end

        subgraph UTILS["Independent Utilities"]
            TRANSLATE["mcp-contextforge-translate<br/>🐍 Protocol Bridge<br/>📦 PyPI | 🐳 Container | 💾 Binaries"]
            WRAPPER["mcp-contextforge-wrapper<br/>🐍 MCP Client Wrapper<br/>📦 PyPI | 🐳 Container"]
            PROXY["mcp-contextforge-reverse-proxy<br/>🐍 NAT/Firewall Traversal<br/>📦 PyPI | 🐳 Container"]
        end

        subgraph PLUGINS["Plugin Ecosystem"]
            PYPLUGINS["mcp-contextforge-plugins-python<br/>🐍 40 Plugins + Framework<br/>📦 PyPI: 40+ packages"]
            RUSTPLUGINS["mcp-contextforge-plugins-rust<br/>🦀 High-Performance Plugins<br/>📦 PyPI: PyO3 wheels"]
        end

        subgraph SERVERS["MCP Servers"]
            PYSERVERS["mcp-contextforge-mcp-servers-python<br/>🐍 4 Servers<br/>📦 PyPI | 🐳 Containers"]
            GOSERVERS["mcp-contextforge-mcp-servers-go<br/>🐹 5 Servers<br/>💾 Binaries | 🐳 Distroless"]
            RUSTSERVERS["mcp-contextforge-mcp-servers-rust<br/>🦀 Future Servers<br/>💾 Static Binaries | 🐳 Alpine"]
        end

        subgraph AGENTS["Agent Runtimes"]
            RUNTIMES["mcp-contextforge-agent-runtimes<br/>🐍 LangChain + Future<br/>📦 PyPI | 🐳 Containers"]
        end

        subgraph INFRA["Infrastructure"]
            HELM["mcp-contextforge-helm<br/>☸️ Kubernetes Helm Charts<br/>📊 OCI Registry | Artifact Hub"]
            DEPLOY["mcp-contextforge-deployment-scripts<br/>⚙️ k8s, Terraform, Ansible<br/>Nginx, Docker Compose"]
        end

        subgraph DOCS["Documentation"]
            DOCSREPO["mcp-contextforge-docs<br/>📝 Markdown<br/>MkDocs Material<br/>🌐 Github Pages"]
        end
    end

    %% Dependencies
    UI -->|depends on| GATEWAY
    PYPLUGINS -->|depends on| GATEWAY
    RUSTPLUGINS -->|depends on| GATEWAY
    TRANSLATE -.->|zero deps| NULL1[" "]
    WRAPPER -.->|zero deps| NULL2[" "]
    PROXY -.->|zero deps| NULL3[" "]
    PYSERVERS -.->|zero deps| NULL4[" "]
    GOSERVERS -.->|zero deps| NULL5[" "]
    RUSTSERVERS -.->|zero deps| NULL6[" "]
    RUNTIMES -->|may use| GATEWAY
    HELM -->|deploys| GATEWAY
    HELM -->|deploys| UI
    HELM -->|deploys| PYPLUGINS
    DEPLOY -->|manages| GATEWAY
    DOCSREPO -->|documents| GATEWAY
    DOCSREPO -->|documents| PYPLUGINS
    DOCSREPO -->|documents| SERVERS

    %% Styling
    classDef coreStyle fill:#e1f5fe,stroke:#01579b,stroke-width:2px
    classDef utilStyle fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef pluginStyle fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef serverStyle fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
    classDef agentStyle fill:#fce4ec,stroke:#880e4f,stroke-width:2px
    classDef infraStyle fill:#fff9c4,stroke:#f57f17,stroke-width:2px
    classDef docStyle fill:#e0f2f1,stroke:#004d40,stroke-width:2px
    classDef nullStyle fill:none,stroke:none

    class GATEWAY,UI coreStyle
    class TRANSLATE,WRAPPER,PROXY utilStyle
    class PYPLUGINS,RUSTPLUGINS pluginStyle
    class PYSERVERS,GOSERVERS,RUSTSERVERS serverStyle
    class RUNTIMES agentStyle
    class HELM,DEPLOY infraStyle
    class DOCSREPO docStyle
    class NULL1,NULL2,NULL3,NULL4,NULL5,NULL6 nullStyle

New Organization: `contextforge-org`

14 Repositories:

1. Core Application

mcp-contextforge-gateway-core - FastAPI gateway, services, ORM, CLI
- PyPI: mcp-contextforge-gateway
- Containers: ghcr.io/contextforge-org/mcp-gateway:*
- ~150K lines

2. User Interface

mcp-contextforge-gateway-ui - Admin UI (HTMX + Alpine.js)
- PyPI: mcp-contextforge-gateway-ui (installs as plugin)
- ~11K lines

3. Independent Utilities (3 repos)

mcp-contextforge-translate - Protocol bridge (stdio ↔ SSE ↔ HTTP ↔ gRPC)
- PyPI: mcp-contextforge-translate
- Containers: ghcr.io/contextforge-org/translate:*
- Binaries: Linux, macOS, Windows (optional)
mcp-contextforge-wrapper - MCP client wrapper
- PyPI: mcp-contextforge-wrapper
- Containers: ghcr.io/contextforge-org/wrapper:*
mcp-contextforge-reverse-proxy - NAT/firewall traversal proxy
- PyPI: mcp-contextforge-reverse-proxy
- Containers: ghcr.io/contextforge-org/reverse-proxy:*

4. Plugin Ecosystem (2 repos)

mcp-contextforge-plugins-python - 40 Python plugins + framework
- PyPi: mcp-contextforge-plugin-framework
- PyPI: 40+ individual packages (mcp-contextforge-plugin-{name})
- Meta packages: mcp-contextforge-plugins, mcp-contextforge-plugins-all
mcp-contextforge-plugins-rust - High-performance Rust plugins (PyO3)
- PyPI: mcp-contextforge-plugins-rust (cross-compiled wheels)

5. MCP Servers (3 repos)

mcp-contextforge-mcp-servers-python - 4 Python servers
- PyPI: mcp-server-{name} (per-server packages)
- Containers: Per-server images
mcp-contextforge-mcp-servers-go - 5 Go servers
- Binaries: Cross-compiled Go executables
- Containers: Distroless images (~5-15 MB)
mcp-contextforge-mcp-servers-rust - Rust servers (future)
- Binaries: Statically linked Rust executables
- Containers: Minimal Alpine images (~3-10 MB)

6. Agent Runtimes (1 repo)

mcp-contextforge-agent-runtimes - LangChain + future runtimes
- PyPI: mcp-contextforge-agent-runtime-{name} (per-runtime)
- Containers: Per-runtime images

7. Infrastructure (2 repos)

mcp-contextforge-helm - Kubernetes Helm charts
- Helm Registry (OCI): oci://ghcr.io/contextforge-org/helm-charts/mcp-stack
- Artifact Hub: Public listing
mcp-contextforge-deployment-scripts - k8s, Terraform, Ansible, Nginx, Docker Compose
- Terraform Registry (optional)
- Ansible Galaxy (optional)

8. Documentation (1 repo)

mcp-contextforge-docs - MkDocs Material site (172 MD files)
- Deployed to: https://docs.contextforge.dev/ or GitHub Pages

9. Desktop Application

mcp-context-forge-desktop - Desktop Electron App

10. CLI

**mcp-contextforge-ctl - CLI Application

11. Google Chrome Plugin

12. Firefox Plugin

13. IDE Integration

Benefits

✅ Modularity

Each component independently versioned
Plugins evolve separately from core
MCP servers have zero core dependencies

✅ Packaging Flexibility

50+ PyPI packages (gateway, plugins, servers, tools)
20+ container images (multi-arch: amd64, arm64)
Helm charts (OCI registry + Artifact Hub)
Go/Rust binaries (cross-compiled executables)

✅ Team Scalability

Clear ownership per repository
CODEOWNERS for focused teams
Parallel development without conflicts

✅ Discoverability

Standalone tools (translate, wrapper, reverse-proxy) independently discoverable
Plugin marketplace with individual packages
MCP server catalog clearly separated

✅ Security

Signed releases (GPG, cosign)
SLSA Build Level 3 provenance
OIDC authentication (no long-lived tokens)
Automated dependency scanning (Renovate, Snyk, Trivy)
SBOM generation

Package Summary

Category	Count	Examples
PyPI Packages	~50+	`mcp-contextforge-gateway`, `mcp-contextforge-plugin-`, `mcp-server-`
Container Images	~20+	`mcp-gateway:`, `translate:`, `mcp-server-:`
Helm Charts	1	`mcp-stack` (OCI + Artifact Hub)
Binaries	~15+	Go/Rust servers, Python tools (optional)

Versioning Strategy

Core Gateway: Independent semver (e.g., v0.8.0)
UI: Follows core version
Plugins (Python): Per-plugin semver (e.g., pii-filter-v1.2.0)
Plugins (Rust): Follows core version
Standalone Tools: Independent semver (e.g., translate-v1.0.0)
MCP Servers: Per-server semver (e.g., docx-v1.0.0)
Helm Chart: Chart version + app version (e.g., Chart: 1.0.0, App: 0.8.0)

Cross-Repository Dependencies

Plugins → Core

# Requires: pip install mcp-contextforge-gateway>=0.7.0
from mcpgateway.models import Tool, Server
from mcpgateway.plugins.framework import PluginBase

UI → Core

# Requires: pip install mcp-contextforge-gateway>=0.8.0
from mcpgateway.services import *

Standalone Tools

Zero gateway dependencies (fully independent)
Minimal deps: httpx, websockets, MCP SDK

MCP Servers

Zero gateway dependencies
Only MCP SDK required

Implementation Checklist

Prerequisites

[ ] Create GitHub organization: contextforge-org
[ ] Setup PyPI projects (~50 packages)
[ ] Setup GHCR repositories
[ ] Setup Helm OCI registry
[ ] Configure organization secrets (OIDC tokens)

Repository Creation

[ ] Core: mcp-contextforge-gateway-core
[ ] UI: mcp-contextforge-gateway-ui
[ ] Independent Modules (3):
- [ ] mcp-contextforge-translate
- [ ] mcp-contextforge-wrapper
- [ ] mcp-contextforge-reverse-proxy
[ ] Plugins (2):
- [ ] mcp-contextforge-plugins-python
- [ ] mcp-contextforge-plugins-rust
[ ] Servers (3):
- [ ] mcp-contextforge-mcp-servers-python
- [ ] mcp-contextforge-mcp-servers-go
- [ ] mcp-contextforge-mcp-servers-rust
[ ] Agent Runtimes:
- [ ] mcp-contextforge-agent-runtimes
[ ] Infrastructure (2):
- [ ] mcp-contextforge-helm
- [ ] mcp-contextforge-deployment-scripts
[ ] Documentation:
- [ ] mcp-contextforge-docs

CI/CD Setup

[ ] GitHub Actions workflows per repo
[ ] PyPI publishing (OIDC trusted publishing)
[ ] Container builds (multi-arch)
[ ] Helm chart publishing (OCI)
[ ] Binary builds (Go/Rust cross-compilation)
[ ] Documentation site deployment

Migration

[ ] Extract code to new repos
[ ] Setup cross-repo imports
[ ] Migrate tests
[ ] Update documentation
[ ] Archive monorepo (read-only)
[ ] Publish first releases

Communication

[ ] Blog post announcing split
[ ] Update README with migration guide
[ ] Update package registries (PyPI, GHCR, Artifact Hub)
[ ] Notify community

Questions for Discussion

GitHub org name: Confirm contextforge-org or alternative?
PyPI namespace: Use mcp-contextforge-* prefix for all packages?
Timeline: When to start migration?
Team ownership: Who owns each repository?
Deprecation: How long to support monorepo before archiving?
Stars: Inheriting github stars.

Oct 25 '25 10:10 crivetimihai

mcp-context-forge mcp-context-forge copied to clipboard

[CHORE] Proposal: Split Monorepo into Separate Repositories

Monorepo Split Proposal

Summary

Current State

Proposed Structure

New Organization: contextforge-org

1. Core Application

2. User Interface

3. Independent Utilities (3 repos)

4. Plugin Ecosystem (2 repos)

5. MCP Servers (3 repos)

6. Agent Runtimes (1 repo)

7. Infrastructure (2 repos)

8. Documentation (1 repo)

9. Desktop Application

10. CLI

11. Google Chrome Plugin

12. Firefox Plugin

13. IDE Integration

Benefits

✅ Modularity

✅ Packaging Flexibility

✅ Team Scalability

✅ Discoverability

✅ Security

Package Summary

Versioning Strategy

Cross-Repository Dependencies

Plugins → Core

UI → Core

Standalone Tools

MCP Servers

Implementation Checklist

Prerequisites

Repository Creation

CI/CD Setup

Migration

Communication

Questions for Discussion

mcp-context-forge
mcp-context-forge copied to clipboard

New Organization: `contextforge-org`