supabase
Introspection Timeout after upgrade from 1.2 to 1.5
Hi, we upgraded our project to Postgres version 15.1.1.19 in the cloud dashboard. From the update, also the ph_graphql extension was updated from 1.2 to 1.5.
Since the update, the introspection does not work anymore and times out.
The used query is
query IntrospectionQuery {
__schema {
queryType {
name
}
mutationType {
name
}
subscriptionType {
name
}
types {
...FullType
}
directives {
name
description
locations
args {
...InputValue
}
}
}
}
fragment FullType on __Type {
kind
name
description
fields(includeDeprecated: true) {
name
description
args {
...InputValue
}
type {
...TypeRef
}
isDeprecated
deprecationReason
}
inputFields {
...InputValue
}
interfaces {
...TypeRef
}
enumValues(includeDeprecated: true) {
name
description
isDeprecated
deprecationReason
}
possibleTypes {
...TypeRef
}
}
fragment InputValue on __InputValue {
name
description
type {
...TypeRef
}
defaultValue
}
fragment TypeRef on __Type {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
}
}
}
}
}
}
}
}
Our project ID is fcttvteqyojvmvjgjyxe
Also in the GraphiQL Editor in the cloud dashboard, no queries are working when impersonating a user. The returning message is
{
"message": "Gateway Timeout"
}
When executing it with the service_role, the queries are workingl.
Introspection in Postman etc. is also not working (I am using the service_key in the request)
The DB logs show for the Postman request
I also tried to increase the response timeout:
But the query times out after a few seconds.
When I execute the introspection query in sql (the query from the logs), it returns after ~20s (which is much slower than in version 1.2 but it is in the response timeframe)
WITH pgrst_source AS (SELECT pgrst_call.pgrst_scalar FROM (SELECT '{
"query": "\n query IntrospectionQuery {\n __schema {\n \n queryType { name }\n mutationType { name }\n subscriptionType { name }\n types {\n ...FullType\n }\n directives {\n name\n description\n \n locations\n args {\n ...InputValue\n }\n }\n }\n }\n\n fragment FullType on __Type {\n kind\n name\n description\n \n fields(includeDeprecated: true) {\n name\n description\n args {\n ...InputValue\n }\n type {\n ...TypeRef\n }\n isDeprecated\n deprecationReason\n }\n inputFields {\n ...InputValue\n }\n interfaces {\n ...TypeRef\n }\n enumValues(includeDeprecated: true) {\n name\n description\n isDeprecated\n deprecationReason\n }\n possibleTypes {\n ...TypeRef\n }\n }\n\n fragment InputValue on __InputValue {\n name\n description\n type { ...TypeRef }\n defaultValue\n \n \n }\n\n fragment TypeRef on __Type {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n ofType {\n kind\n name\n }\n }\n }\n }\n }\n }\n }\n }\n }\n }\n ",
"operationName": "IntrospectionQuery"
}'::json AS json_data) pgrst_payload, LATERAL (SELECT CASE WHEN json_typeof(pgrst_payload.json_data) = 'array' THEN pgrst_payload.json_data ELSE json_build_array(pgrst_payload.json_data) END AS val) pgrst_uniform_json, LATERAL (SELECT "query" FROM json_to_recordset(pgrst_uniform_json.val) AS _("query" text) LIMIT 1) pgrst_body , LATERAL (SELECT "graphql_public"."graphql"("query" := pgrst_body."query") pgrst_scalar) pgrst_call) SELECT null::bigint AS total_result_set, 1 AS page_total, coalesce(json_agg(_postgrest_t.pgrst_scalar)->0, 'null') AS body, nullif(current_setting('response.headers', true), '') AS response_headers, nullif(current_setting('response.status', true), '') AS response_status, '' AS response_inserted FROM (SELECT "graphql".* FROM "pgrst_source" AS "graphql" ) _postgrest_t
Before the update, it was working perfect, the introspection finished after a few seconds.
To Reproduce Steps to reproduce the behavior:
- Copy the introspection query from above or go to Postman
- Make a introspection query to the project
- query times out
Versions:
- PostgreSQL: 15.1.1.19
- pg_graphql: 1.5
Additional context Using the cloud dashboard
Thanks @PhilippS93 for reporting, I'm looking into this to pinpoint which commit introduced the performance problem.
This is a performance regression introduced with the field merging feature.
We have reverted that PR and cut a pre-release v1.5.1-mergeless available here that resolves the issue at the cost of dropping support for field merging.
A rewrite of the field merging logic is being worked on and will be in place before cutting the official 1.5.1 release.
If you are a Supabase user experiencing this issue, please:
- open a support ticket at https://supabase.com/support
- reference this issue
- requesting an upgrade to pg_graphql
v1.5.1-mergeless
the update requires 0 downtime
sorry for the inconvenience
Hey @PhilippS93 we have tracked the cause of the slowdown to PR #472 which we have reverted for now. This PR implemented a feature to merge duplicate fields in queries so that feature would also be reverted. We'll implement it again in a separate PR with better performance. I'm working on a new release with this fix.
Hey @imor @olirice , thank you! The v1.5.1-mergeless fix is working perfectly!
We're facing this issue too, and while we wait for 1.5.1 to be released, is it possible to downgrade the version of pg_graphql in our local development environment?
Installing an older version of supabase and reinstalling the images with supabase stop && supabase start doesn't seem to affect the extension version on localhost:54323.
@bryanmylee could you please raise a ticket for this. v1.5.1 is already released on GH but is not yet part of our Postgres image so I'll manually upgrade your project to 1.5.1.
@imor Thanks imor! I've submitted a ticket, but just to clarify, we also need to update our local environment, since we depend on it for schema introspection.
@bryanmylee was the upgrade done for you? If not, can you please share the ticket id and I'll take a look.
@imor We've manually upgraded the projects to 1.5.1 since it was merged into the main Supabase branch, thanks!
However, our local development version is still stuck on 1.5.0.
Our ticket's been resolved! For others who might want to upgrade their local Supabase environment directly, you can modify the files in supabase/.temp:
echo "15.1.1.24" > supabase/.temp/postgres-version
Then run supabase stop && supabase start.
If the extensions aren't upgraded, you can delete the container's volumes and restart Supabase again.