cardano-graphql icon indicating copy to clipboard operation
cardano-graphql copied to clipboard

Published 20 hours ago • verified publisher icon input-output-hk

cardano-graphql-801-cardano-db-sync can't use snapshot

Open zhy827827 opened this issue 10 months ago • 2 comments

Ask a question

cardano-graphql-8.0.1$ cat start.sh
export NETWORK=mainnet &&\
docker compose pull &&\
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz \
docker compose up -d &&\
docker compose logs -f


cardano-graphql-8.0.1$ sudo docker logs -f cardano-graphql-801-cardano-db-sync-1
/run/secrets
Generating PGPASS file
Connecting to network specified in configuration.yaml
Downloading snapshot https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 52.0G  100 52.0G    0     0  95.3M      0  0:09:18  0:09:18 --:--:-- 94.9M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   121  100   121    0     0    355      0 --:--:-- --:--:-- --:--:--   355


db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz: OK
db/
db/4128.dat.gz
db/4125.dat.gz
db/4046.dat.gz
db/4127.dat.gz
db/4152.dat.gz
db/4050.dat.gz
db/4109.dat.gz
db/4115.dat.gz
db/4083.dat.gz
db/4062.dat.gz
db/4072.dat.gz
db/4036.dat.gz
db/4130.dat.gz
db/4081.dat.gz
db/4070.dat.gz
db/4087.dat.gz
db/4058.dat.gz
db/4132.dat.gz
db/4068.dat.gz
db/4142.dat.gz
db/4064.dat.gz
db/4123.dat.gz
db/4060.dat.gz
db/4093.dat.gz
db/4097.dat.gz
db/4040.dat.gz
db/4113.dat.gz
db/4042.dat.gz
db/4038.dat.gz
db/4078.dat.gz
db/4150.dat.gz
db/4085.dat.gz
db/4111.dat.gz
db/4091.dat.gz
db/4105.dat.gz
db/4134.dat.gz
db/4119.dat.gz
db/4144.dat.gz
db/4138.dat.gz
db/toc.dat
db/4089.dat.gz
db/4066.dat.gz
db/4101.dat.gz
db/4156.dat.gz
db/4140.dat.gz
db/4079.dat.gz
db/4044.dat.gz
db/4136.dat.gz
db/4099.dat.gz
db/4117.dat.gz
db/4146.dat.gz
db/4048.dat.gz
db/4154.dat.gz
db/4054.dat.gz
db/4056.dat.gz
db/4052.dat.gz
db/4121.dat.gz
db/4076.dat.gz
db/4074.dat.gz
db/4034.dat.gz
db/4148.dat.gz
db/4107.dat.gz
db/4103.dat.gz
db/4095.dat.gz
119404784-058c10851a-473.lstate.gz
/nix/store/1bys3yjgm97yxy91sfy8f5law41fyif9-postgresql-setup.sh: line 194: sed: command not found

I copy sed from the host to the container, but there is still a problem bash: /bin/sed: cannot execute: required file not found

version: cardano-graphql-8.0.1 ghcr.io/intersectmbo/cardano-db-sync:13.2.0.1 cardanosolutions/cardano-node-ogmios:v6.1.0_8.7.3-mainnet

how to Fix this problem?

zhy827827 avatar Mar 28 '24 02:03 zhy827827

This seems like a db-sync issue, I'll discuss it with these devs and come back to you as soon as possible.

Kammerlo avatar Mar 28 '24 06:03 Kammerlo

@zhy827827 Just to let you know the ticket is open here at db-sync. https://github.com/IntersectMBO/cardano-db-sync/pull/1662

Kammerlo avatar Apr 16 '24 11:04 Kammerlo

I encountered the same problem, how to solve it, version 8.0.3

vino-evan avatar May 09 '24 08:05 vino-evan

It seems, it's not included in the current release: https://github.com/IntersectMBO/cardano-db-sync/commit/9f4653ed28b4e44b9c5e9f8fca3b7f380e152eab#commitcomment-141727357

I'll track this issue and keep you updated.

Kammerlo avatar May 12 '24 10:05 Kammerlo

You can use this version of the docker image：

docker pull ghcr.io/intersectmbo/cardano-db-sync:13.2.0.2-docker

vino-evan avatar May 14 '24 03:05 vino-evan

This version is also used in 8.0.3 of graphQL. Does ist solve the issue for you?

Kammerlo avatar May 14 '24 04:05 Kammerlo

hello @Kammerlo , i used 8.0.3 of graphQL,solve the issue,Thank you!

But there is another issues: {"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T03:32:38.288Z","v":0}

The Asset calculation was always 1% , unchanged

zhy827827 avatar May 20 '24 03:05 zhy827827

Are there any other log outputs in another service? How do you start it?

Kammerlo avatar May 20 '24 06:05 Kammerlo

I used the startup script

export NETWORK=mainnet &&\
docker compose pull &&\
#RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10272678-x86_64.tgz \
docker compose up -d &&\
docker compose logs -f

Not using snapshots, synchronizing from Genesis block.There are no other error log here.

docker logs -f cardano-graphql-803-server-1 --tail 100

{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:52:53.248Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:52:53.250Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625243,"id":"0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c"},"msg":"[Object: null prototype] {\n  slot: 124625243,\n  id: '0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c'\n}","time":"2024-05-20T07:52:53.256Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:08.979Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:08.981Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625243,"id":"0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c"},"msg":"[Object: null prototype] {\n  slot: 124625243,\n  id: '0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c'\n}","time":"2024-05-20T07:53:08.988Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:24.852Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:24.854Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625306,"id":"6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55"},"msg":"[Object: null prototype] {\n  slot: 124625306,\n  id: '6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55'\n}","time":"2024-05-20T07:53:24.862Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:40.693Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:40.695Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625306,"id":"6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55"},"msg":"[Object: null prototype] {\n  slot: 124625306,\n  id: '6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55'\n}","time":"2024-05-20T07:53:40.698Z","v":0}

docker logs -f cardano-graphql-803-background-1 --tail 100

{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"a80b9489e5a8e5e5eadcc0c2856f46c5883f5c3d88676ce5559d8cdc","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.975Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"bb9cf64bfca2c8611aefb81b5341ec5f7dba39ec2b6464a4e8a13e51","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.980Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"f37e16045a5aa79503c1e3a36c0892a04a6f4471e4a097c9c98d3854","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.985Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"aebcb6eaba17dea962008a9d693e39a3160b02b5b89b1c83e537c599","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.990Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"8cf66d8c74af3aa53ba498b0d3f18a21f7ed3d12128850bb2b61146d","msg":"Adding metadata to asset","time":"2024-05-20T08:00:08.382Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"dc1c91e3afb2f7ad8a837eb0fe4c75f7250f968d55f4e426e4ec2ff4","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.622Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"8cf66d8c74af3aa53ba498b0d3f18a21f7ed3d12128850bb2b61146d","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.635Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"dc1c91e3afb2f7ad8a837eb0fe4c75f7250f968d55f4e426e4ec2ff4","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.639Z","v":0}

curl   -X POST   -H "Content-Type: application/json"   -d '{"query": "{ cardanoDbMeta { initialized syncPercentage }}"}'   http://localhost:3200/graphql |jq
{
  "data": {
    "cardanoDbMeta": {
      "initialized": true,
      "syncPercentage": 100
    }
  }
}

zhy827827 avatar May 20 '24 07:05 zhy827827

For how long did you wait regarding the Assets? Because sometimes it can take sometime until the percentage rises further, due to syncing speed.

Kammerlo avatar May 20 '24 08:05 Kammerlo

I have been waiting for three days now

zhy827827 avatar May 20 '24 08:05 zhy827827

I'll check it

Kammerlo avatar May 21 '24 05:05 Kammerlo

hello @Kammerlo
Is there any progress on this matter?Thank you for your contribution to the community

zhy827827 avatar May 22 '24 02:05 zhy827827

I'm still on it. Did your node reach the tip?

Kammerlo avatar May 22 '24 09:05 Kammerlo

yes.new tip slot Is normally synchronized

zhy827827 avatar May 22 '24 09:05 zhy827827

fyi, I'm currently doing some tests. An issue could be the limit of requests in direction of the tokenregistry. Next week I'm planning to change the way we are fetching assets from graphql, then this won't be an issue anymore.

Kammerlo avatar May 22 '24 12:05 Kammerlo

Hey @zhy827827 I tried it on my end on mainnet. I ended up with Assets at 100%, even while the node didn't reach the tip yet. I even added extra parsing safety for Blocks in chainfollower (which is responsible to fill the asset table). Could you try again with this branch https://github.com/cardano-foundation/cardano-graphql/pull/872?

If it doesn't help I would propose two options to get closer to solve your problem:

  1. Try a resync with the new branch
  2. We can have a call and look into the issue together, if you like

Kammerlo avatar May 23 '24 06:05 Kammerlo

Thank you very much.My Asset are still in sync at 1%。 I will try again using the new branch.But I want to know how you deployment graphql.

Here are all the images I have used:

614f25b19a53   cardanofoundation/cardano-graphql-background:8.0.1-mainnet   "node background.js"     7 days ago    Up 3 hours                                                                                        cardano-graphql-803-background-1
99bbdb814597   cardanofoundation/cardano-graphql-hasura:8.0.1               "docker-entrypoint.s…"   7 days ago    Up 7 days             0.0.0.0:8099->8080/tcp, :::8099->8080/tcp                                   cardano-graphql-803-hasura-1
0899d44d2b9d   ghcr.io/intersectmbo/cardano-db-sync:13.2.0.2                "/nix/store/1fk383dn…"   7 days ago    Up 3 hours                                                                                        cardano-graphql-803-cardano-db-sync-1
323dd0be8dcc   postgres:14.10-alpine                                        "docker-entrypoint.s…"   7 days ago    Up 7 days             0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                   cardano-graphql-803-postgres-1
a049eacec6ae   cardanofoundation/cardano-graphql-server:8.0.1-mainnet       "node index.js"          7 days ago    Up 6 days             3200/tcp, 0.0.0.0:3200->3100/tcp, :::3200->3100/tcp                         cardano-graphql-803-server-1
09142ee1b887   cardanosolutions/cardano-node-ogmios:v6.2.0_8.9.0-mainnet    "/tini -g -- /root/c…"   7 days ago    Up 7 days (healthy)   3000/tcp, 12788/tcp, 0.0.0.0:1337->1337/tcp, :::1337->1337/tcp, 12798/tcp   cardano-graphql-803-cardano-node-ogmios-1

zhy827827 avatar May 23 '24 06:05 zhy827827

I just started it with sudo docker compose up -d --build Without passing anything regarding the network mainnet is chosen as a default.

Kammerlo avatar May 23 '24 06:05 Kammerlo

Closing this issue due to inactivity. If you have any other issues @zhy827827 let me know or reopen this issue

Kammerlo avatar Jul 04 '24 09:07 Kammerlo