cardano-graphql icon indicating copy to clipboard operation
cardano-graphql copied to clipboard

Issue with DB

Open ctscobra opened this issue 3 years ago • 6 comments

On multiple nodes we are seeing these issues , what are they and what is the fix ?

postgres_1                  | 2021-11-14 09:45:59.503 UTC [30384] ERROR:  value too long for type character varying(65536)
postgres_1                  | 2021-11-14 09:45:59.503 UTC [30384] STATEMENT:  WITH "Asset__mutation_result_alias" AS (UPDATE "public"."Asset" SET "decimals" = NULL,"description" = ($1)::text,"logo" = ($2)::text,"metadataHash" = ($3)::bpchar,"name" = ($4)::text,"ticker" = NULL,"url" = NULL  WHERE (('true') AND ((((("public"."Asset"."assetId") = (($5)::bytea)) AND ('true')) AND ('true')) AND ('true'))) RETURNING * , CASE WHEN 'true' THEN NULL ELSE "hdb_catalog"."check_violation"('update check constraint failed')  END ), "Asset__all_columns_alias" AS (SELECT  "assetId" , "assetName" , "decimals" , "description" , "fingerprint" , "firstAppearedInSlot" , "logo" , "metadataHash" , "name" , "policyId" , "ticker" , "url"  FROM "Asset__mutation_result_alias"      ) SELECT  json_build_object('affected_rows', (SELECT  COUNT(*)  FROM "Asset__all_columns_alias"      ), 'returning', (SELECT  coalesce(json_agg("root" ), '[]' ) AS "root" FROM  (SELECT  row_to_json((SELECT  "_1_e"  FROM  (SELECT  "_0_root.base"."assetId" AS "assetId"       ) AS "_1_e"      ) ) AS "root" FROM  (SELECT  *  FROM "Asset__all_columns_alias" WHERE ('true')     ) AS "_0_root.base"      ) AS "_2_root"      ) )
postgres_1                  | 2021-11-14 09:53:22.406 UTC [30417] ERROR:  value too long for type character varying(65536)
postgres_1                  | 2021-11-14 09:53:22.406 UTC [30417] STATEMENT:  WITH "Asset__mutation_result_alias" AS (UPDATE "public"."Asset" SET "decimals" = NULL,"description" = ($1)::text,"logo" = ($2)::text,"metadataHash" = ($3)::bpchar,"name" = ($4)::text,"ticker" = ($5)::text,"url" = ($6)::text  WHERE (('true') AND ((((("public"."Asset"."assetId") = (($7)::bytea)) AND ('true')) AND ('true')) AND ('true'))) RETURNING * , CASE WHEN 'true' THEN NULL ELSE "hdb_catalog"."check_violation"('update check constraint failed')  END ), "Asset__all_columns_alias" AS (SELECT  "assetId" , "assetName" , "decimals" , "description" , "fingerprint" , "firstAppearedInSlot" , "logo" , "metadataHash" , "name" , "policyId" , "ticker" , "url"  FROM "Asset__mutation_result_alias"      ) SELECT  json_build_object('affected_rows', (SELECT  COUNT(*)  FROM "Asset__all_columns_alias"      ), 'returning', (SELECT  coalesce(json_agg("root" ), '[]' ) AS "root" FROM  (SELECT  row_to_json((SELECT  "_1_e"  FROM  (SELECT  "_0_root.base"."assetId" AS "assetId"       ) AS "_1_e"      ) ) AS "root" FROM  (SELECT  *  FROM "Asset__all_columns_alias" WHERE ('true')     ) AS "_0_root.base"      ) AS "_2_root"      ) )

ctscobra avatar Nov 14 '21 12:11 ctscobra

Is this on the mainnet? Do you have logs from cardano-graphql that reveal the asset triggering the DB violation?

rhyslbw avatar Nov 15 '21 05:11 rhyslbw

Yes this is on mainet . Atm i have the logs from new instances that are trying to sync but we do get this on all.

postgres_1                  | 2021-11-15 06:25:48.882 UTC [17834] ERROR:  value too long for type character varying(65536)
postgres_1                  | 2021-11-15 06:25:48.882 UTC [17834] STATEMENT:  WITH "Asset__mutation_result_alias" AS (UPDATE "public"."Asset" SET "decimals" = ($1)::integer,"description" = ($2)::text,"logo" = ($3)::text,"metadataHash" = ($4)::bpchar,"name" = ($5)::text,"ticker" = ($6)::text,"url" = NULL  WHERE (('true') AND ((((("public"."Asset"."assetId") = (($7)::bytea)) AND ('true')) AND ('true')) AND ('true'))) RETURNING * , CASE WHEN 'true' THEN NULL ELSE "hdb_catalog"."check_violation"('update check constraint failed')  END ), "Asset__all_columns_alias" AS (SELECT  "assetId" , "assetName" , "decimals" , "description" , "fingerprint" , "firstAppearedInSlot" , "logo" , "metadataHash" , "name" , "policyId" , "ticker" , "url"  FROM "Asset__mutation_result_alias"      ) SELECT  json_build_object('affected_rows', (SELECT  COUNT(*)  FROM "Asset__all_columns_alias"      ), 'returning', (SELECT  coalesce(json_agg("root" ), '[]' ) AS "root" FROM  (SELECT  row_to_json((SELECT  "_1_e"  FROM  (SELECT  "_0_root.base"."assetId" AS "assetId"       ) AS "_1_e"      ) ) AS "root" FROM  (SELECT  *  FROM "Asset__all_columns_alias" WHERE ('true')     ) AS "_0_root.base"      ) AS "_2_root"      ) )
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Notice:64] [2021-11-15 06:25:47.77 UTC] Identity Starting Subscription Worker, valency 1
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Notice:27603] [2021-11-15 06:25:47.78 UTC] Identity Connection Attempt Start, destination LocalAddress "/node-ipc/node.socket"
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Notice:27603] [2021-11-15 06:25:47.78 UTC] Identity Connection Attempt End, destination LocalAddress "/node-ipc/node.socket" outcome: ConnectSuccessLast
cardano-db-sync-extended_1  | [db-sync-node.Handshake:Info:27603] [2021-11-15 06:25:47.78 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"}) Send (ClientAgency TokPropose,MsgProposeVersions (fromList [(NodeToClientV_1,TInt 764824073),(NodeToClientV_2,TInt 764824073),(NodeToClientV_3,TInt 764824073),(NodeToClientV_4,TInt 764824073),(NodeToClientV_5,TInt 764824073),(NodeToClientV_6,TInt 764824073),(NodeToClientV_7,TInt 764824073),(NodeToClientV_8,TInt 764824073),(NodeToClientV_9,TInt 764824073)]))
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Notice:64] [2021-11-15 06:25:47.80 UTC] Identity Required subscriptions started
cardano-db-sync-extended_1  | [db-sync-node.Handshake:Info:27603] [2021-11-15 06:25:47.82 UTC] WithMuxBearer (ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"}) Recv (ServerAgency TokConfirm,MsgAcceptVersion NodeToClientV_9 (TInt 764824073))
cardano-db-sync-extended_1  | [db-sync-node.Mux:Info:27603] [2021-11-15 06:25:47.82 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"} event: Handshake Client end, duration 0.041411334s
cardano-db-sync-extended_1  | [db-sync-node.Mux:Info:27604] [2021-11-15 06:25:47.82 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"} event: State: Mature
cardano-db-sync-extended_1  | [db-sync-node:Info:27607] [2021-11-15 06:25:47.82 UTC] Starting chainSyncClient
cardano-db-sync-extended_1  | [db-sync-node:Error:27607] [2021-11-15 06:25:47.82 UTC] ChainSyncWithBlocksPtcl: libpq: failed (no connection to the server
cardano-db-sync-extended_1  | [db-sync-node.Mux:Info:27604] [2021-11-15 06:25:47.82 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"} event: State: Dead
cardano-db-sync-extended_1  | [db-sync-node.Mux:Notice:27604] [2021-11-15 06:25:47.82 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "/ipc/node.socket"} event: Miniprotocol MiniProtocolNum 5 InitiatorDir terminated with exception libpq: failed (no connection to the server
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Error:27603] [2021-11-15 06:25:47.82 UTC] Identity Application Exception: LocalAddress "/node-ipc/node.socket" libpq: failed (no connection to the server
cardano-db-sync-extended_1  | [db-sync-node.ErrorPolicy:Error:6] [2021-11-15 06:25:47.82 UTC] IP LocalAddress "/node-ipc/node.socket" ErrorPolicyUnhandledApplicationException libpq: failed (no connection to the server
cardano-db-sync-extended_1  | [db-sync-node.Subscription:Notice:64] [2021-11-15 06:25:49.06 UTC] Identity Restarting Subscription after 1.281090556s desired valency 1 current valency 0
cardano-node-ogmios_1       | [4ccd5950:cardano.node.LocalErrorPolicy:Error:2019] [2021-11-15 06:25:48.86 UTC] IP LocalAddress "" ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed "<socket: 30> closed when reading data, waiting on next header True")

ctscobra avatar Nov 15 '21 06:11 ctscobra

I see the same error. Any news on this?

CherryDT avatar Jan 11 '22 10:01 CherryDT

any update on this ?

ctscobra avatar Feb 20 '22 09:02 ctscobra

I am also receiving this error with cardano-db-sync 12.0.2.

centromere avatar May 19 '22 14:05 centromere

@centromere are you running https://github.com/input-output-hk/cardano-graphql/releases/tag/6.2.0 ? That release included an update to align with the token metadata ticker length increase, which could be the root cause here.

rhyslbw avatar Jul 06 '22 08:07 rhyslbw