mcp-context-forge icon indicating copy to clipboard operation
mcp-context-forge copied to clipboard
verified publisher icon IBM

[Feature Request]: Migrate from JWT Tokens to Short Opaque API Tokens

Open kevalmahajan opened this issue 3 weeks ago • 0 comments

🧭 Type of Feature

Please select the most appropriate category:

  • [x] Enhancement to existing functionality
  • [ ] New feature or capability
  • [ ] New MCP-compliant server
  • [ ] New component or integration
  • [ ] Developer tooling or test improvement
  • [ ] Packaging, automation and deployment (ex: pypi, docker, quay.io, kubernetes, terraform)
  • [ ] Other (please describe below)

🧭 Epic

Title: Migrate from Long JWT Tokens to Short Opaque API Tokens for Enhanced Security and Usability

Goal: Replace the current JWT-based authentication system with short, opaque API tokens (format: cf-XXXX) to eliminate information exposure, prevent cross-environment token reuse, and improve user experience with shorter, more manageable tokens.

Why now: The current system has critical security vulnerabilities and usability issues that affect all API users:

  1. JWT tokens expose sensitive information (email, team IDs, permissions, admin status) that can be decoded by anyone without the secret key​
  2. Development environment tokens can be used in production if JWT secret keys are shared across environments
  3. No environment isolation mechanism exists to prevent cross-environment token abuse
  4. Current JWT tokens are 700+ characters long, making them difficult to copy, paste, and display in CLI tools and UIs​
  5. Industry leaders (GitHub, GitLab, Stripe, Slack) use short, prefixed tokens for better developer experience​
  6. Opaque tokens are recommended for scenarios requiring enhanced security and revocability​
  7. Token prefixes improve secret scanning accuracy and reduce false positives to 0.5%​
  8. Modern API security practices (2025) emphasize minimal data exposure and instant revocation capabilities

🙋♂️ User Story 1:

As a: Platform Security Administrator I want: Environment-specific token prefixes that prevent cross-environment token usage So that: Development tokens cannot be accidentally or maliciously used in staging/production environments

✅ Acceptance Criteria

Scenario: Development environment generates dev-prefixed tokens
  Given the platform is running in "development" environment
  And JWT_ENVIRONMENT is set to "dev"
  When a user creates a new API token
  Then the token has prefix "cf-"
  And the token format is "cf-<random_22_chars>"
  And the environment is stored in the token metadata

Scenario: Staging environment generates stage-prefixed tokens
  Given the platform is running in "staging" environment
  And JWT_ENVIRONMENT is set to "stage"
  When a user creates a new API token
  Then the token has prefix "cfstg-"
  And the token format is "cfstg-<random_22_chars>"

Scenario: Production environment generates prod-prefixed tokens
  Given the platform is running in "production" environment
  And JWT_ENVIRONMENT is set to "prod"
  When a user creates a new API token
  Then the token has prefix "cfprd-"
  And the token format is "cfprd-<random_22_chars>"

Scenario: Cross-environment token usage is blocked
  Given I have a development token "cf-abc123def456ghi789"
  And the platform is running in "production" environment
  When I attempt to authenticate with the development token
  Then authentication fails with HTTP 401
  And the error message indicates "Token environment mismatch: development token used in production"
  And a security event is logged

🙋♂️ User Story 2

As a: Platform Developer using MCP Gateway APIs I want: Short, opaque API tokens instead of long JWT strings So that: I can easily copy/paste tokens, use them in CI/CD pipelines, and avoid exposing sensitive information in logs or error messages

✅ Acceptance Criteria

Scenario: User creates a new API token
  Given I am authenticated as an active team member
  And I have "tokens.create" permission
  When I POST to /api/v1/tokens with token name and scope
  Then the system generates a short API token with format "cf-<random_22_chars>"
  And the API token is stored with a hash in the database
  And the API token maps to an internal JWT token
  And the response includes the API token (shown only once)
  And the JWT token is never exposed to the user

Scenario: User authenticates with new API token
  Given I have a valid API token "cf-k7R9fQw2VDp3Zx4mN8tJ1sB0uHcYrLq"
  When I send a request with "Authorization: Bearer cf-k7R9fQw2VDp3Zx4mN8tJ1sB0uHcYrLq"
  Then the system looks up the token hash in the database
  And retrieves the associated JWT token claims
  And authenticates me with the mapped user identity and permissions
  And updates the last_used timestamp

Scenario: API token cannot be decoded to reveal information
  Given I have an API token "cf-k7R9fQw2VDp3Zx4mN8tJ1sB0uHcYrLq"
  When I attempt to decode or inspect the token
  Then I cannot extract email, server IDs, permissions, or other sensitive data
  And the token remains opaque

🙋♂️ User Story 3

As a: Existing API Consumer with JWT tokens I want: Backward compatibility with my existing JWT-based authentication So that: The applications continue to work while I migrate to the new API token format

✅ Acceptance Criteria

Scenario: JWT token authentication continues to work
  Given I have a valid JWT token from before the API token migration
  When I send a request with "Authorization: Bearer <jwt_token>"
  Then the system recognizes it as a JWT token (starts with "eyJ")
  And authenticates me using the existing JWT validation flow
  And the request succeeds

Scenario: JWT token authentication includes deprecation warning
  Given I authenticate using a JWT token
  When the authentication succeeds
  Then a deprecation warning header is included: "X-Auth-Deprecation: JWT tokens are deprecated. Please migrate to API tokens."
  And the response includes a Link header pointing to migration documentation
  And my authentication continues to work normally

Scenario: JWT token phaseout timeline
  Given the system is in the deprecation period
  When users authenticate with JWT tokens
  Then JWT authentication works for 90 days from feature release
  And deprecation warnings are logged
  And after 90 days, JWT tokens return HTTP 401 with migration instructions

📐 Design Sketch (optional)

Current Authentication Flow:

sequenceDiagram
    participant User
    participant API
    participant Auth
    participant DB
    
    User->>API: POST /tokens (create token)
    API->>Auth: generate_token()
    Auth->>Auth: Create JWT with payload<br/>(email, teams, permissions)
    Auth->>Auth: Sign with JWT_SECRET_KEY
    Auth-->>API: Long JWT token (500+ chars)
    API->>DB: Store hash(JWT), jti, metadata
    API-->>User: Return JWT token (ONCE)
    
    Note over User: User stores and uses JWT
    
    User->>API: Request + Bearer JWT
    API->>Auth: Decode JWT (no secret needed!)
    Note over Auth: ⚠️ Email, teams, permissions EXPOSED
    Auth->>Auth: Verify signature with secret
    Auth->>DB: Check jti in revocation table
    Auth-->>API: EmailUser object
    API-->>User: Response

Proposed New Authentication Flow:

sequenceDiagram
    participant User
    participant API
    participant Auth
    participant DB
    
    User->>API: POST /tokens (create token)
    API->>Auth: generate_api_token()
    Auth->>Auth: Generate random string (32 chars)
    Auth->>Auth: Create token: "cf-" + random
    Auth->>Auth: Create internal JWT for metadata
    Auth->>DB: Store hash(api_token), hash(jwt), jti, metadata
    API-->>User: Return "cf-k7R9..." (35 chars)
    
    Note over User: User stores short API token
    
    User->>API: Request + Bearer cf-k7R9...
    API->>Auth: Detect token type (starts with "cf-")
    Auth->>Auth: Hash incoming token
    Auth->>DB: Lookup by token_hash
    DB-->>Auth: Token metadata (jti, user, teams, scopes)
    Auth->>DB: Check expiration & revocation
    Auth-->>API: EmailUser object
    API-->>User: Response
    
    Note over Auth,DB: ✅ Token is opaque<br/>✅ Instant revocation<br/>✅ Environment-specific

Token Format Comparison:

graph TD
    A[Token Types] --> B[Legacy JWT Token]
    A --> C[New API Token]
    
    B --> B1[Length: 500-2000 chars]
    B --> B2[Format: eyJ...]
    B --> B3[Decodable: YES ⚠️]
    B --> B4[Contains: email, teams,<br/>permissions, admin status]
    B --> B5[Revocation: Delayed<br/>until expiration]
    
    C --> C1[Length: ~35 chars]
    C --> C2[Format: cf-XXXX]
    C --> C3[Decodable: NO ✅]
    C --> C4[Contains: Nothing<br/>opaque reference]
    C --> C5[Revocation: Immediate<br/>database lookup]
    
    style B3 fill:#ff6b6b,color:#fff
    style B4 fill:#ff6b6b,color:#fff
    style B5 fill:#ff6b6b,color:#fff
    style C3 fill:#51cf66,color:#fff
    style C4 fill:#51cf66,color:#fff
    style C5 fill:#51cf66,color:#fff

🔗 MCP Standards Check

  • [ ] Change adheres to current MCP specifications
  • [ ] No breaking changes to existing MCP-compliant integrations
  • [ ] If deviations exist, please describe them below:

🔄 Alternatives Considered

Alternative 1: Encrypted JWT Tokens (JWE)

Description: Use JSON Web Encryption to encrypt JWT payload Pros:

  • Standards-based (RFC 7516)
  • Maintains JWT ecosystem compatibility
  • Payload encrypted but still self-contained

Cons:

  • Tokens still very long (600+ chars with encryption overhead)​
  • Adds cryptographic complexity
  • Still cannot prevent cross-environment reuse if keys shared
  • Cannot be instantly revoked without additional blacklist mechanism

Why rejected: Doesn't solve the primary issues (length, cross-environment reuse, revocation complexity)

Alternative 2: Short-lived JWT + Refresh Tokens Description: Issue very short-lived JWTs (5-15 min) with separate refresh tokens Pros:

  • Limits exposure window of compromised JWT
  • Industry-standard pattern​
  • Automatic rotation

Cons:

  • Still exposes data in JWT payload during validity period
  • Adds refresh token management complexity
  • Doesn't solve cross-environment issue
  • Users must implement refresh logic in applications

Why rejected: Adds complexity without addressing core security issues of data exposure

Alternative 3: Keep JWT, Add Token Prefixes Only Description: Continue using JWT but add environment-specific prefixes Pros:

  • Minimal code changes
  • Easy migration

Cons:

  • Doesn't solve data exposure problem
  • Tokens still very long
  • Revocation still delayed
  • Only addresses environment isolation partially

Why rejected: Solves only 1 of 3 major problems

📓 Additional Context

Include related issues, links to discussions, issues, etc.

kevalmahajan avatar Nov 18 '25 05:11 kevalmahajan