hub-monorepo icon indicating copy to clipboard operation
hub-monorepo copied to clipboard

Hubble continously printing error after Snapshot Sync

Open SamuelLHuber opened this issue 5 months ago • 0 comments

What is the bug? After downloading the snapshot and trying to sync with the Snapshot from HOYT the hub is stuck printing the following message

 {"message":" ████████████████████████████████████████ 100% | Getting snapshot | 3900/3900 | ETA: LLs","timestamp":"2024-09-06T09:00:30.454Z","type":"err","process_id":0,"app_name":"hubble"}                                                     │
│ {"message":"\n","timestamp":"2024-09-06T09:00:30.455Z","type":"err","process_id":0,"app_name":"hubble"} 

How can it be reproduced? (optional) Include steps, code samples, replits, screenshots and anything else that would be helpful to reproduce the problem.

Run the official docker image with the following command (unless you have your identity already created, then just run it normally. Env variables used are attached below.

image:
          repository: farcasterxyz/hubble
          tag: 1.15.0
        command: ['/bin/sh', '-c']
        args:
          - |
            set -e
            node build/cli.js identity create
            npx pm2-runtime start pm2.config.cjs

Additional context (optional) Add any other context about the problem here.

ENV Variables

 env:
          - name: CATCHUP_SYNC_WITH_SNAPSHOT
            value: "true"
          - name: NODE_OPTIONS
            value: "--no-warnings --max-old-space-size=8192"
          - name: HUBBLE_ARGS
            value: >-
              start
              --ip 0.0.0.0
              --gossip-port 2282
              --rpc-port 2283
              --eth-mainnet-rpc-url https://rpc.ankr.com/eth
              --l2-rpc-url https://rpc.ankr.com/optimism
              --network 1
              --hub-operator-fid 16085
              --rpc-subscribe-per-ip-limit 4
              -b /dns/hoyt.farcaster.xyz/tcp/2282
              --statsd-metrics-server hubble-statsd.onchain.svc.cluster.local:8125
              --opt-out-diagnostics true

HTTPApi Info endpoint

curl http://127.0.0.1:2281/v1/info?dbstats=1
{"version":"1.15.0","isSyncing":false,"nickname":"Farcaster Hub","rootHash":"779a28bcfa9f1acb1056d53954aa000d74ad6526","dbStats":{"numMessages":539304179,"numFidEvents":851581,"numFnameEvents":663272,"approxSize":167876139398},"peerId":"12D3KooWHx1mcNJaXHGgRJj6q2PdH3kYxkewN5CtYemEW5upbeCC","hubOperatorFid":16085}

the num Messages keeps increasing, so it may still be syncing and not fully caught up

More context of the build up / discussion can be found on Farcaster: https://www.supercast.xyz/c/0xc1088677a8ae256d7a9ce1199e3c6daee986e0e7

SamuelLHuber avatar Sep 06 '24 09:09 SamuelLHuber