parse-server icon indicating copy to clipboard operation
parse-server copied to clipboard

Graphql introspection with masterkey does not work when NODE_ENV is production

Open Stovedal opened this issue 1 month ago • 20 comments

New Issue Checklist

Issue Description

The GQL introspection endpoint is blocked for environments with NODE_ENV=production even when requesting it with masterkey.

As I understand it from this PR introspection should always work when masterKey is used, but since the introspection boolean on ApolloServer defaults to false the request is blocked even though it's not stopped by the introspection plugin.

Steps to reproduce

Run server without setting graphQLPublicIntrospection to true, set NODE_ENV to production and run an introspection request with master key.

Actual Outcome

{"errors":[{"message":"GraphQL introspection is not allowed by Apollo Server, but the query contained __schema or __type. To enable introspection, pass introspection: true to ApolloServer in production","locations":[{"line":2,"column":3}],"extensions":{"validationErrorCode":"INTROSPECTION_DISABLED","code":"GRAPHQL_VALIDATION_FAILED"}}]}

Expected Outcome

< introspection result >

Environment

7.5.3

Server

  • Parse Server version: 7.5.3
  • Operating system: MacOS
  • Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Heroku

Database

  • System (MongoDB or Postgres): Mongo
  • Database version: 7
  • Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): MongoDB Atlas

Client

  • SDK (iOS, Android, JavaScript, PHP, Unity, etc): FILL_THIS_OUT
  • SDK version: FILL_THIS_OUT

Logs

Stovedal avatar Nov 03 '25 16:11 Stovedal

🚀 Thanks for opening this issue!

❌ Please fill out all fields with a placeholder FILL_THIS_OUT. If a field does not apply to the issue, fill in n/a or delete the line.

Hijacking to suggest adding the possibility to have more granular control of allowing introspection or not by passing a function into the graphql configuration options.

This function would have access to the request context and return a boolean.

A use case for this would be to have a special key as passed as a separate header which has lower privileges than the master key. This would allow for more broad access to introspection without granting access to the master key or having public access.

emil19 avatar Nov 04 '25 15:11 emil19

Good idea. Regarding master key, the docs say:

All the schemas methods require MasterKey.

Therefore I'd say the master key should also always be allowed to use introspection. We may need to extend the access scopes table with a new "Schema" column since it's meta data and we haven't defined that yet.

Scope Internal data Read-only data (1) Custom data Schema Restricted by CLP, ACL Key
Internal r/w r/w r/w r/w no maintenanceKey
Master -/- r/- r/w r/w no masterKey
ReadOnlyMaster -/- r/- r/- r/- no readOnlyMasterKey
Session -/- r/- r/w -/- yes sessionToken

mtrezza avatar Nov 04 '25 21:11 mtrezza

i think it will be fixed in the apollo update PR, it's weird because we have test cases

Moumouls avatar Nov 07 '25 21:11 Moumouls

A use case for this would be to have a special key as passed as a separate header which has lower privileges than the master key. This would allow for more broad access to introspection without granting access to the master key or having public access.

@emil19 An even better idea could be to expose the full Apollo Server configuration as a Parse Server option.

That way, Parse Server would simply pass the configuration to Apollo, and wouldn’t need to maintain or define specific features itself.

@mtrezza The issue, however, is that any breaking changes in Apollo Server would also result in breaking changes for Parse Server.

I’m not sure Parse Server is ready yet for a full inversion of control, where we could pass an ApolloServer instance into the ParseGraphQLServer.

Moumouls avatar Nov 07 '25 21:11 Moumouls

The issue, however, is that any breaking changes in Apollo Server would also result in breaking changes for Parse Server.

I think it depends on risks and benefits of that approach. I don't see an issue with exposing apollo, if it saves us maintenance efforts. The breaking changes need to be addressed anyway, to keep apollo on latest major versions, if I followed your thoughts correctly. What would that mean for DX? I suppose the GraphQL endpoint would then operate outside some of Parse Server's security mechanisms?

mtrezza avatar Nov 07 '25 22:11 mtrezza

@Stovedal the bug will be fixed here: https://github.com/parse-community/parse-server/pull/9888

I improved the coverage of the introspection protection system.

Moumouls avatar Nov 07 '25 22:11 Moumouls

@mtrezza I haven’t yet investigated the full impact of using Inversion of Control (IoC) by injecting an Apollo configuration or instance through Parse Server. However, it could be a great approach to reduce maintenance and expose Apollo’s full set of features directly to developers, there’s really no better approach than IoC.

The only disadvantage of this approach is that we would support IoC specifically for Apollo. The goal would not be to support every GraphQL framework. From my point of view, this isn’t an issue, as Apollo is the most mature and well-maintained GraphQL server available.

Moumouls avatar Nov 07 '25 22:11 Moumouls

@mtrezza I haven’t yet investigated the full impact of using Inversion of Control (IoC) by injecting an Apollo configuration or instance through Parse Server. However, it could be a great approach to reduce maintenance and expose Apollo’s full set of features directly to developers, there’s really no better approach than IoC.

The only disadvantage of this approach is that we would support IoC specifically for Apollo. The goal would not be to support every GraphQL framework. From my point of view, this isn’t an issue, as Apollo is the most mature and well-maintained GraphQL server available.

@coderabbitai What would be the implication of this suggestion?

mtrezza avatar Nov 08 '25 18:11 mtrezza

[!TIP] For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Find the ParseGraphQLServer file
fd -e js -e ts ParseGraphQLServer

# Get the current Apollo Server setup in ParseGraphQLServer
ast-grep --pattern 'new ApolloServer($$$)'

# Look at the graphQL configuration options
rg -n "graphQL" --type=ts --type=js -g "Options.js" -C5

# Find where Apollo configuration is defined
rg -n "apolloConfig|ApolloServerOptions" --type=ts --type=js -C3

Length of output: 1050


🏁 Script executed:

#!/bin/bash
# Read the ParseGraphQLServer.js file to understand current implementation
cat src/GraphQL/ParseGraphQLServer.js

# Find the IntrospectionControlPlugin
rg -n "IntrospectionControlPlugin" --type=js -C10

# Find graphQL configuration in Options
rg -n "graphQL" src/Options/index.js -A5 -B2

# Look for where graphQL options are defined in the docs
fd -e md -e js Config | xargs rg -l "graphQLPublicIntrospection" 2>/dev/null | head -5

Length of output: 11892


🏁 Script executed:

#!/bin/bash
# Look at the current Parse Server configuration structure for GraphQL
cat src/Options/index.js | grep -A 30 "graphQL"

# Find the ParseGraphQLController to understand how configuration is managed
cat src/Controllers/ParseGraphQLController.js | head -100

# Look for existing middleware and security patterns
rg -n "handleParseHeaders|handleParseSession|handleParseErrors" src/middlewares -A5 -B2

# Check if there are any tests showing the security model
rg -n "isMaster.*graphql" spec/ --type=js | head -20

Length of output: 6575


🏁 Script executed:

#!/bin/bash
# Find middleware files
fd middleware

# Search for where handleParseHeaders is defined
rg -n "export.*handleParseHeaders" --type=js -A20

# Look at how auth is set up in middleware
rg -n "req.auth.*isMaster|isMaster.*req.auth" --type=js -C3

# Find where Apollo Server dependencies are used
cat package.json | jq '.dependencies | with_entries(select(.key | contains("apollo") or contains("graphql")))'

# Check for any existing configuration extension patterns
rg -n "apolloServerOptions|apolloConfig" --type=js -C5

Length of output: 8986


@mtrezza Based on my analysis of the codebase, here are the implications of exposing Apollo Server configuration through IoC:

Architecture Implications

Current Architecture:

  • Parse Server creates ApolloServer with specific configuration including CSRF prevention, introspection settings, custom plugins, and schema
  • A custom IntrospectionControlPlugin checks isMaster or isMaintenance flags from the request context before allowing introspection
  • Security context is established through Parse Server's middleware chain that runs before Apollo

IoC Approach Implications:

1. Security Model Preservation

The most critical concern is maintaining Parse Server's security model:

  • The middleware chain (handleParseHeadershandleParseSessionhandleParseErrors) must execute before Apollo to set up req.auth.isMaster, session tokens, etc.
  • The custom IntrospectionControlPlugin relies on Parse-specific auth context that wouldn't exist if users provide their own Apollo instance
  • CSRF prevention is configured specifically for Parse headers (X-Parse-Application-Id)

2. Breaking Changes Propagation

  • Apollo Server breaking changes would directly impact Parse Server users
  • However, this risk exists anyway when upgrading Apollo versions
  • Could be mitigated by providing sensible defaults and clear migration guides

3. Schema Generation Coupling

  • Parse Server regenerates Apollo instance when schema changes, which is tightly coupled to ParseGraphQLSchema
  • Full IoC would complicate this hot-reload behavior

4. Developer Experience Trade-offs

Pros:

  • Access to all Apollo features (custom plugins, formatError, various Apollo options)
  • Reduced Parse Server maintenance for Apollo-specific features
  • Flexibility for advanced use cases (custom authentication, rate limiting via Apollo plugins)

Cons:

  • Developers must understand Apollo Server deeply
  • Risk of misconfiguration breaking Parse Server's security guarantees
  • Increased support burden for Parse Server maintainers

Recommended Approaches

Option 1: Partial Configuration Exposure (Incremental)

{
  graphQLPath: '/graphql',
  apolloServerOptions: {
    // Merge with Parse Server defaults
    introspection: customIntrospectionFunction,
    plugins: [customPlugin, ...parseDefaultPlugins],
    formatError: customFormatter,
    // But don't allow overriding schema, context, or csrfPrevention
  }
}

Option 2: Plugin System

Expose hooks at specific points rather than full config:

{
  graphQLPath: '/graphql',
  apolloPlugins: [customPlugin],
  introspectionCheck: (context) => boolean,
  formatGraphQLError: (error, context) => formatted
}

Option 3: Full IoC (Advanced Users Only)

{
  customApolloServer: async (schema, parseContext) => {
    // User creates their own Apollo instance
    // Parse Server only provides schema and context helper
    return apolloInstance;
  }
}

Specific Concern: Introspection Control

For the original issue, a simpler solution would be adding an introspection callback:

{
  graphQLIntrospection: (context) => {
    // Access to req.auth, custom headers, etc.
    return context.auth?.isMaster || customCheck(context);
  }
}

This addresses the use case (custom keys for introspection) without full IoC complexity.

Recommendation

Start with Option 2 (Plugin System) as it:

  • Solves the immediate use case elegantly
  • Maintains Parse Server's security model
  • Provides extensibility without full Apollo exposure
  • Allows incremental expansion of capabilities
  • Keeps breaking change risk contained

Full IoC (Option 3) could be added later as an "advanced mode" with clear warnings about security implications.

Would you like me to search for examples of how other frameworks handle similar GraphQL server abstraction challenges?


🧠 Learnings used
Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9858
File: src/GraphQL/ParseGraphQLServer.js:176-178
Timestamp: 2025-09-21T15:43:32.265Z
Learning: The GraphQL playground feature in ParseGraphQLServer.js (applyPlayground method) is intended for development environments only, which is why it includes the master key in client-side headers.

coderabbitai[bot] avatar Nov 08 '25 18:11 coderabbitai[bot]

i think @mtrezza the option 1 is the best option, and error is thrown if a developer try to provide an option managed by parse-server

Moumouls avatar Nov 08 '25 20:11 Moumouls

@Stovedal the bug will be fixed here: #9888

I improved the coverage of the introspection protection system.

Great! Looking forward to the update 👍

Stovedal avatar Nov 10 '25 08:11 Stovedal

@mtrezza @Moumouls

While we're talking about changing the apollo server instance I'd like to highlight another issue we have with the current setup.

As the apollo server is created upon request instead of on server start with default behavior we create and start ~100 apollo servers in the time window of getting the first requests and the server actually being set. This messed a lot with our startup request times. We've had to modify the parse server code directly to mitigate it.

With Option 3: Full IoC, we could avoid this completely as we'd have control over start and create of the server, but if we want to keep to option 1 or 2 I'd appreciate the possibility to configure the start behavior of the server (or just create and start it on server startup — I can't really see the benefit of creating it upon request)

Stovedal avatar Nov 10 '25 09:11 Stovedal

@Stovedal, can you explain your use case? Parse Server SHOULD only computes the server once, so maybe the server mutex should be set before the await (apollo.start()), ensuring that only one Apollo Server instance is initialized while parallel requests wait for the first instance to be ready.

This might actually be the real source of your issue.

Parse Server’s GraphQL implementation is special because the Apollo instance gets invalidated whenever a schema change is detected. Since Parse Server is schemaless by default, the Apollo instance is recreated whenever the Parse.Schema is modified. There’s no real workaround for this, but the process is quite fast.

Could you explain what kind of fix you applied on your side?

I believe this might just be a mutex issue.

Based on my understanding and @coderabbitai’s report, the best option for now is to expose only a partial interface of Apollo Server.

async _getServer() {
    const schemaRef = this.parseGraphQLSchema.graphQLSchema;
    const newSchemaRef = await this.parseGraphQLSchema.load();
    if (schemaRef === newSchemaRef && this._server) {
      return this._server;
    }
    const { schema, context } = await this._getGraphQLOptions();
    const apollo = new ApolloServer({
      csrfPrevention: {
        // See https://www.apollographql.com/docs/router/configuration/csrf/
        // needed since we use graphql upload
        requestHeaders: ['X-Parse-Application-Id'],
      },
      introspection: this.config.graphQLPublicIntrospection,
      plugins: [ApolloServerPluginCacheControlDisabled(), IntrospectionControlPlugin(this.config.graphQLPublicIntrospection)],
      schema,
    });
    await apollo.start();
    this._server = expressMiddleware(apollo, {
      context,
    });
    return this._server;
  }

Tested feature


  describe('_getServer', () => {
    it('should only return new server on schema changes', async () => {
      parseGraphQLServer.server = undefined;
      const server1 = await parseGraphQLServer._getServer();
      const server2 = await parseGraphQLServer._getServer();
      expect(server1).toBe(server2);

      // Trigger a schema change
      const obj = new Parse.Object('SomeClass');
      await obj.save();

      const server3 = await parseGraphQLServer._getServer();
      const server4 = await parseGraphQLServer._getServer();
      expect(server3).not.toBe(server2);
      expect(server3).toBe(server4);
    });
  });

Moumouls avatar Nov 10 '25 16:11 Moumouls

[!TIP] For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Read the current ParseGraphQLServer implementation
cat src/GraphQL/ParseGraphQLServer.js

# Search for any mutex or locking mechanism in the codebase
rg -n "mutex|lock|semaphore" --type=js -g "ParseGraphQL*" -C3

# Look for how _getServer is called
rg -n "_getServer" src/GraphQL/ParseGraphQLServer.js -C5

Length of output: 8423


@Moumouls You're absolutely correct about the mutex issue! I examined the current implementation and can confirm there is no locking mechanism in the _getServer() method.

The Race Condition

Here's what happens during startup with concurrent requests:

async _getServer() {
    const schemaRef = this.parseGraphQLSchema.graphQLSchema;
    const newSchemaRef = await this.parseGraphQLSchema.load();
    if (schemaRef === newSchemaRef && this._server) {  // ← All concurrent requests see undefined
      return this._server;
    }
    // ... all requests proceed to create Apollo instances ...
    await apollo.start();  // ← Multiple instances starting in parallel
    this._server = expressMiddleware(apollo, { context });  // ← Last one wins
    return this._server;
}

Timeline of the race condition:

  1. Request 1 arrives → this._server is undefined → starts creating Apollo
  2. Request 2 arrives → this._server is undefined (not set yet) → starts creating Apollo
  3. Requests 3-100 arrive → same issue
  4. All 100 requests call await apollo.start() concurrently
  5. Only the last assignment to this._server is kept, but 100 instances were created

Recommended Fix

Add a promise-based mutex to ensure only one initialization happens:

async _getServer() {
    // If initialization is in progress, wait for it
    if (this._serverInitializing) {
      await this._serverInitializing;
      return this._server;
    }

    const schemaRef = this.parseGraphQLSchema.graphQLSchema;
    const newSchemaRef = await this.parseGraphQLSchema.load();
    if (schemaRef === newSchemaRef && this._server) {
      return this._server;
    }

    // Set the mutex BEFORE any async operations
    this._serverInitializing = (async () => {
      try {
        const { schema, context } = await this._getGraphQLOptions();
        const apollo = new ApolloServer({
          csrfPrevention: {
            requestHeaders: ['X-Parse-Application-Id'],
          },
          introspection: this.config.graphQLPublicIntrospection,
          plugins: [
            ApolloServerPluginCacheControlDisabled(), 
            IntrospectionControlPlugin(this.config.graphQLPublicIntrospection)
          ],
          schema,
        });
        await apollo.start();
        this._server = expressMiddleware(apollo, { context });
      } finally {
        this._serverInitializing = null;
      }
    })();

    await this._serverInitializing;
    return this._server;
}

This ensures:

  • First request sets this._serverInitializing promise
  • Concurrent requests see the promise and wait for it
  • Only one Apollo instance is created
  • All requests get the same initialized server

@Stovedal - This should resolve the issue you described. Would be great to hear what fix you applied to confirm this is the same problem you encountered!


🧠 Learnings used
Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9858
File: src/GraphQL/ParseGraphQLServer.js:176-178
Timestamp: 2025-09-21T15:43:32.265Z
Learning: The GraphQL playground feature in ParseGraphQLServer.js (applyPlayground method) is intended for development environments only, which is why it includes the master key in client-side headers.

coderabbitai[bot] avatar Nov 10 '25 16:11 coderabbitai[bot]

[!TIP] For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Get the actual current implementation of ParseGraphQLServer.js
cat src/GraphQL/ParseGraphQLServer.js

Length of output: 7706


@Moumouls You're spot-on about the mutex issue! I've examined the current implementation and can confirm there's no locking mechanism preventing multiple Apollo Server instances from being created during concurrent requests.

The Race Condition Confirmed

Looking at the current code in src/GraphQL/ParseGraphQLServer.js:

async _getServer() {
    const schemaRef = this.parseGraphQLSchema.graphQLSchema;
    const newSchemaRef = await this.parseGraphQLSchema.load();
    if (schemaRef === newSchemaRef && this._server) {  // ← All concurrent requests see undefined
      return this._server;
    }
    const { schema, context } = await this._getGraphQLOptions();
    const apollo = new ApolloServer({ /* ... */ });
    await apollo.start();  // ← Multiple instances starting in parallel
    this._server = expressMiddleware(apollo, { context });  // ← Last write wins
    return this._server;
}

What happens during startup:

  1. 100 concurrent requests arrive
  2. All see this._server === undefined
  3. All proceed to create and start Apollo instances
  4. 100 instances are initialized (expensive!)
  5. Only the last assignment is kept

Recommended Mutex Implementation

Set the mutex before any async operations:

async _getServer() {
    // If initialization is in progress, wait for it
    if (this._serverInitializing) {
      await this._serverInitializing;
      return this._server;
    }

    const schemaRef = this.parseGraphQLSchema.graphQLSchema;
    const newSchemaRef = await this.parseGraphQLSchema.load();
    if (schemaRef === newSchemaRef && this._server) {
      return this._server;
    }

    // Set mutex BEFORE any async work
    this._serverInitializing = (async () => {
      try {
        const { schema, context } = await this._getGraphQLOptions();
        const apollo = new ApolloServer({
          csrfPrevention: {
            requestHeaders: ['X-Parse-Application-Id'],
          },
          introspection: this.config.graphQLPublicIntrospection,
          plugins: [
            ApolloServerPluginCacheControlDisabled(), 
            IntrospectionControlPlugin(this.config.graphQLPublicIntrospection)
          ],
          schema,
        });
        await apollo.start();
        this._server = expressMiddleware(apollo, { context });
      } finally {
        this._serverInitializing = null;  // Clear mutex
      }
    })();

    await this._serverInitializing;
    return this._server;
}

Enhanced Test Case

Your test case is great, but consider adding an explicit concurrent request test:

describe('_getServer', () => {
  it('should only return new server on schema changes', async () => {
    parseGraphQLServer._server = undefined;
    const server1 = await parseGraphQLServer._getServer();
    const server2 = await parseGraphQLServer._getServer();
    expect(server1).toBe(server2);

    // Trigger a schema change
    const obj = new Parse.Object('SomeClass');
    await obj.save();

    const server3 = await parseGraphQLServer._getServer();
    const server4 = await parseGraphQLServer._getServer();
    expect(server3).not.toBe(server2);
    expect(server3).toBe(server4);
  });

  it('should handle concurrent requests without creating multiple servers', async () => {
    parseGraphQLServer._server = undefined;
    parseGraphQLServer._serverInitializing = null;
    
    // Simulate 10 concurrent requests
    const serverPromises = Array(10).fill(null).map(() => 
      parseGraphQLServer._getServer()
    );
    
    const servers = await Promise.all(serverPromises);
    
    // All should return the same instance
    servers.forEach(server => {
      expect(server).toBe(servers[0]);
    });
  });
});

This fix should resolve @Stovedal's startup performance issue and is worth implementing regardless of the partial Apollo config exposure decision.

Looking forward to hearing what fix Stovedal applied to confirm this analysis!


🧠 Learnings used
Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9858
File: src/GraphQL/ParseGraphQLServer.js:176-178
Timestamp: 2025-09-21T15:43:32.265Z
Learning: The GraphQL playground feature in ParseGraphQLServer.js (applyPlayground method) is intended for development environments only, which is why it includes the master key in client-side headers.

Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9445
File: spec/ParseLiveQuery.spec.js:1340-1375
Timestamp: 2025-05-09T09:59:06.289Z
Learning: New tests in the parse-server repository should use async/await with promise-based patterns rather than callback patterns with `done()`. The preferred pattern is to create a Promise that resolves when an expected event occurs, then await that Promise.

Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9445
File: spec/ParseLiveQuery.spec.js:1340-1375
Timestamp: 2025-05-09T09:59:06.289Z
Learning: Tests in the parse-server repository should use promise-based approaches rather than callback patterns with `done()`. Use a pattern where a Promise is created that resolves when the event occurs, then await that promise.

Learnt from: mtrezza
Repo: parse-community/parse-server PR: 9445
File: spec/ParseLiveQuery.spec.js:1312-1338
Timestamp: 2025-05-04T20:41:05.147Z
Learning: New tests in the parse-server repository should use async/await with promise-based patterns rather than callback patterns with `done()`.

coderabbitai[bot] avatar Nov 10 '25 16:11 coderabbitai[bot]

@Stovedal i'm currently investigating the apollo server start up process, i'll push a dedicated PR

Moumouls avatar Nov 10 '25 17:11 Moumouls

@Stovedal i'm currently investigating the apollo server start up process, i'll push a dedicated PR

That's great! Yes the mutex is the root issue to the issues we've had with startup, and how we solved it was more a workaround than a fix. Basically, we added a call to _getServer in applyGraphql

  async applyGraphQL(app) {
    if (!app || !app.use) {
      requiredParameter('You must provide an Express.js app instance!');
    }

    await this._getServer(); // run _getServer to intialise the server straight away

    app.use(this.config.graphQLPath, corsMiddleware());

and then awaited applyGraphQL before opening the port and starting to take requests.

Stovedal avatar Nov 10 '25 17:11 Stovedal

@Stovedal here the fix :) i successfully reproduced the issue, and fixed it properly https://github.com/parse-community/parse-server/pull/9929

Moumouls avatar Nov 10 '25 17:11 Moumouls

@Stovedal here the fix :) i successfully reproduced the issue, and fixed it properly #9929

Hey, that's great, quick work! Looking forward to the release — one less mod on our side.

Stovedal avatar Nov 11 '25 07:11 Stovedal