inspector icon indicating copy to clipboard operation
inspector copied to clipboard

[wip] Client auth compatibility checker

Open pcarleton opened this issue 4 months ago • 2 comments

TODO

cd auth-compat
npx tsx src/cli/index.ts --command "npx tsx examples/typescript-client/test-client.ts" --suite metadata
❯ npx tsx src/cli/index.ts --command "npx tsx examples/typescript-client/test-client.ts" --suite metadata
Running MCP compliance tests...

Running test suite: Metadata Location Tests
Description: Tests different OAuth protected resource metadata locations
============================================================

▶ Standard location with WWW-Authenticate
EXIT CODE: 0
EXIT CODE: 0
  ✅ PASS

▶ Non-standard location with WWW-Authenticate
EXIT CODE: 0
EXIT CODE: 0
  ✅ PASS

▶ Nested well-known path with WWW-Authenticate
EXIT CODE: 0
EXIT CODE: 0
  ✅ PASS

▶ Standard location without WWW-Authenticate
EXIT CODE: 0
EXIT CODE: 0
  ✅ PASS

▶ Non-standard location without WWW-Authenticate
EXIT CODE: 1
EXIT CODE: 1
  ✅ PASS

============================================================
Suite Summary: Metadata Location Tests
  Passed: 5/5

============================================================
OVERALL SUMMARY
============================================================
Total Suites Passed: 1/1

✅ All test suites passed!

cd auth-compat
npx tsx src/cli/index.ts --command "npx tsx examples/typescript-client/test-client.ts" --test 'Nested well-known path with WWW-Authenticate' --verbose


Running MCP compliance tests...
Found 1 test(s) matching "Nested well-known path with WWW-Authenticate"


Running test suite: Metadata Location Tests
Description: Tests different OAuth protected resource metadata locations
============================================================

▶ Nested well-known path with WWW-Authenticate
[AUTH SERVER] Started on port 54552
[VALIDATION SERVER] Auth server started at http://localhost:54552
[VALIDATION SERVER] Started on port 54553 (stateless mode)
  Server started at: http://localhost:54553/mcp
  Metadata URL: http://localhost:54553/.well-known/oauth-protected-resource/mcp
EXIT CODE: 0
EXIT CODE: 0
[AUTH SERVER] Stopped
[VALIDATION SERVER] Stopped
  ✅ Result: PASS

  ====== INTERLEAVED HTTP TRACE ======

  --- [MCP SERVER] Request #1 ---
  Timestamp: 2025-08-11T09:24:58.640Z
  POST /mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  content-type: application/json
  accept: application/json, text/event-stream
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 160

  {"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"test-client","version":"1.0.0"}},"jsonrpc":"2.0","id":0}

  HTTP/1.1 401 Unauthorized
  x-powered-by: Express
  www-authenticate: Bearer error="invalid_token", error_description="Missing Authorization header", resource_metadata="http://localhost:54553/.well-known/oauth-protected-resource/mcp"
  content-type: application/json; charset=utf-8
  content-length: 76
  etag: W/"4c-ptrIdu+3yjAtarglCEu6XVLnz2c"

  {"error":"invalid_token","error_description":"Missing Authorization header"}


  --- [MCP SERVER] Request #2 ---
  Timestamp: 2025-08-11T09:24:58.646Z
  GET /.well-known/oauth-protected-resource/mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  mcp-protocol-version: 2025-06-18
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 88
  etag: W/"58-+c1pGzee6l8QA5+6zx/IpEGGbvM"

  {"resource":"http://localhost:54553","authorization_servers":["http://localhost:54552"]}


  --- [AUTH] Request #3 ---
  Timestamp: 2025-08-11T09:24:58.649Z
  GET /.well-known/oauth-authorization-server HTTP/1.1
  host: localhost:54552
  connection: keep-alive
  mcp-protocol-version: 2025-06-18
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 414
  etag: W/"19e-h4oF/mEnp2rKj9YgyCyrHzQQU6o"

  {"issuer":"http://localhost:54552","authorization_endpoint":"http://localhost:54552/authorize","token_endpoint":"http://localhost:54552/token","registration_endpoint":"http://localhost:54552/register","response_types_supported":["code"],"grant_types_supported":["authorization_code","refresh_token"],"code_challenge_methods_supported":["S256"],"token_endpoint_auth_methods_supported":["none","client_secret_post"]}


  --- [AUTH] Request #4 ---
  Timestamp: 2025-08-11T09:24:58.651Z
  POST /register HTTP/1.1
  host: localhost:54552
  connection: keep-alive
  content-type: application/json
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 209

  {"client_name":"Test Client","redirect_uris":["http://localhost:8090/callback"],"grant_types":["authorization_code","refresh_token"],"response_types":["code"],"token_endpoint_auth_method":"none","scope":"mcp"}

  HTTP/1.1 201 Created
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 238
  etag: W/"ee-JWBmj6QS5LSMqQEhnnaFC8KfdY0"

  {"client_id":"test_client_id","client_name":"Test Client","redirect_uris":["http://localhost:8090/callback"],"grant_types":["authorization_code","refresh_token"],"response_types":["code"],"token_endpoint_auth_method":"client_secret_post"}


  --- [AUTH] Request #5 ---
  Timestamp: 2025-08-11T09:24:58.653Z
  GET /authorize?response_type=code&client_id=test_client_id&code_challenge=LDyJXMWDzEtkBCdLZ8Xp0uLSvbJWsP9DNVPBNuwEwD4&code_challenge_method=S256&redirect_uri=http%3A%2F%2Flocalhost%3A8090%2Fcallback&scope=mcp&resource=http%3A%2F%2Flocalhost%3A54553%2F HTTP/1.1
  host: localhost:54552
  connection: keep-alive
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 302 Found
  x-powered-by: Express
  location: http://localhost:8090/callback?code=test_auth_code_123
  vary: Accept
  content-type: text/plain; charset=utf-8
  content-length: 76

  Found. Redirecting to http://localhost:8090/callback?code=test_auth_code_123


  --- [MCP SERVER] Request #6 ---
  Timestamp: 2025-08-11T09:24:58.654Z
  GET /.well-known/oauth-protected-resource/mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  mcp-protocol-version: 2025-06-18
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 88
  etag: W/"58-+c1pGzee6l8QA5+6zx/IpEGGbvM"

  {"resource":"http://localhost:54553","authorization_servers":["http://localhost:54552"]}


  --- [AUTH] Request #7 ---
  Timestamp: 2025-08-11T09:24:58.655Z
  GET /.well-known/oauth-authorization-server HTTP/1.1
  host: localhost:54552
  connection: keep-alive
  mcp-protocol-version: 2025-06-18
  accept: */*
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 414
  etag: W/"19e-h4oF/mEnp2rKj9YgyCyrHzQQU6o"

  {"issuer":"http://localhost:54552","authorization_endpoint":"http://localhost:54552/authorize","token_endpoint":"http://localhost:54552/token","registration_endpoint":"http://localhost:54552/register","response_types_supported":["code"],"grant_types_supported":["authorization_code","refresh_token"],"code_challenge_methods_supported":["S256"],"token_endpoint_auth_methods_supported":["none","client_secret_post"]}


  --- [AUTH] Request #8 ---
  Timestamp: 2025-08-11T09:24:58.657Z
  POST /token HTTP/1.1
  host: localhost:54552
  connection: keep-alive
  content-type: application/x-www-form-urlencoded
  accept: application/json
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 233

  {"grant_type":"authorization_code","code":"test_auth_code_123","code_verifier":"gry4TQlihZo3UAq6MnXmlq72gu4PENmLNUKgicyIpp7","redirect_uri":"http://localhost:8090/callback","client_id":"test_client_id","resource":"http://localhost:54553/"}

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: application/json; charset=utf-8
  content-length: 135
  etag: W/"87-Z6j0f/vQXNxSxaYrVi7M8Wx7Uog"

  {"access_token":"test_access_token_abc","token_type":"Bearer","expires_in":3600,"refresh_token":"test_refresh_token_xyz","scope":"mcp"}


  --- [MCP SERVER] Request #9 ---
  Timestamp: 2025-08-11T09:24:58.658Z
  POST /mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  authorization: Bearer test_access_token_abc
  content-type: application/json
  accept: application/json, text/event-stream
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 160

  {"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"test-client","version":"1.0.0"}},"jsonrpc":"2.0","id":1}

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: text/event-stream
  cache-control: no-cache
  connection: keep-alive

  event: message
data: {"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{"listChanged":true}},"serverInfo":{"name":"validation-server","version":"1.0.0"}},"jsonrpc":"2.0","id":1}




  --- [MCP SERVER] Request #10 ---
  Timestamp: 2025-08-11T09:24:58.664Z
  POST /mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  authorization: Bearer test_access_token_abc
  mcp-protocol-version: 2025-06-18
  content-type: application/json
  accept: application/json, text/event-stream
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 54

  {"method":"notifications/initialized","jsonrpc":"2.0"}

  HTTP/1.1 202 Accepted
  x-powered-by: Express


  --- [MCP SERVER] Request #11 ---
  Timestamp: 2025-08-11T09:24:58.664Z
  GET /mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  authorization: Bearer test_access_token_abc
  mcp-protocol-version: 2025-06-18
  accept: text/event-stream
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate

  HTTP/1.1 405 Method Not Allowed
  x-powered-by: Express

  {"jsonrpc":"2.0","error":{"code":-32000,"message":"Method not allowed in stateless mode"},"id":null}


  --- [MCP SERVER] Request #12 ---
  Timestamp: 2025-08-11T09:24:58.665Z
  POST /mcp HTTP/1.1
  host: localhost:54553
  connection: keep-alive
  authorization: Bearer test_access_token_abc
  mcp-protocol-version: 2025-06-18
  content-type: application/json
  accept: application/json, text/event-stream
  accept-language: *
  sec-fetch-mode: cors
  user-agent: node
  accept-encoding: gzip, deflate
  content-length: 46

  {"method":"tools/list","jsonrpc":"2.0","id":2}

  HTTP/1.1 200 OK
  x-powered-by: Express
  content-type: text/event-stream
  cache-control: no-cache
  connection: keep-alive

  event: message
data: {"result":{"tools":[{"name":"test-tool","title":"Test Tool","description":"A simple test tool for validation","inputSchema":{"type":"object","properties":{"message":{"type":"string","description":"Test message"}},"required":["message"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}}]},"jsonrpc":"2.0","id":2}



  ========================


  [Client Output]
  STDOUT: Connecting to MCP server at: http://localhost:54553/mcp
🔐 OAuth required - handling authorization...
✅ Successfully connected with authentication
✅ Successfully listed tools
✅ Connection closed successfully

  ✅ PASS

============================================================
Suite Summary: Metadata Location Tests
  Passed: 1/1

============================================================
OVERALL SUMMARY
============================================================
Total Suites Passed: 1/1

✅ All test suites passed!

pcarleton avatar Aug 11 '25 09:08 pcarleton

🎭 Playwright E2E Test Results

✅  24 passed

Details

24 tests across 3 suites
 30.7 seconds
 e0a11f8
ℹ️  Test Environment: Ubuntu Latest, Node.js v22.18.0 Browsers: Chromium, Firefox

📊 View Detailed HTML Report (download artifacts)

github-actions[bot] avatar Aug 11 '25 09:08 github-actions[bot]

I think this is super useful. Will test the pr against some example servers to see how things perform.

tobinsouth avatar Aug 11 '25 19:08 tobinsouth