Schema checks not displaying in Studio, despite appearing in GH status checks

[wyattEpp]

In the process of moving to managed federation, we've implemented schema checks in CI flows for multiple services according to these studio docs.

The CLI is receiving expected response for valid/invalid schema update checks, and the pass/fail of checks is getting reported in our service repos' Github status checks via the Apollo Studio GH app (implemented per these studio docs). Despite this, the schema checks are not actually appearing in the Apollo Studio "Checks" tab for each graph variant (link provided in Github status check just links to the Services tab instead of the passing/failing check).

Not sure if this could be related to configuration, or if we're encountering a bug.

[jstjoe]

Hi @wyattEpp, thanks for logging an issue about this. That does sound strange. Could you share a screenshot of what you're seeing in GitHub? The only scenario that comes to mind is when the composition step fails we currently don't show it under Checks. But the fact that you're getting a link to the Studio Services page is odd.

[wyattEpp]

@jstjoe thanks for the fast response! We were expecting to see failures and successes in checks (due to impacted recorded client operations, schema composition failures, safe changes, and 0 changes), but the only checks we've seen so far have been for safe schema additions.

An expected composition failure (extension of non-existent type):

Response to Apollo CLI check in CI:

Github status check with link:

Graph variant's Checks page has no related entry.

[wyattEpp]

Checks supports the ability to override flagged changes per https://www.apollographql.com/docs/studio/schema-checks/#overriding-flagged-changes, which is great, since we're making these Github status checks blocking.

If reporting composition failures within Checks is not possible, but composition failures come back as failed checks in Github status, how can one override expected composition failures within Studio for cases such as https://www.apollographql.com/docs/federation/entities/#migrating-entities-and-fields-advanced?

[jstjoe]

Thanks @wyattEpp this is really helpful. First, it does sound like the issue here is that you're getting Composition errors before we can check for affected operations. Our Checks UI doesn't currently show anything if you attempted to run checks but it failed at the composition step. We're actually changing this right now - you should see the change in the next couple of weeks. With this update we'll actually show distinct steps for Composition and "Operations", and if there's a Composition error we'll show you some details about it as well.

But that doesn't directly solve the override problem you're facing, unfortunately. We don't currently have a built-in way for you to override a composition failure on a check. This is something that we're planning to address soon - we're evaluating some different options now - but in the meantime you'd have to unblock it in GitHub or your CI/CD when performing migrations like this which break composition.

[daniman]

Hey @wyattEpp, sorry for not following up with you on this ages ago! We show composition check failures on Studio's Checks page, so I'm going to close this issue. Thank you for reporting it.