erigon
erigon copied to clipboard
RPC daemon subscription incosistent
System information
Erigon version: git_branch=devel git_tag=v2021.10.03-1907-ga9bde213d-dirty OS & Version: Linux Commit hash : a9bde213d9537bdae31e2a4a7427acf45be8481d Network: mainnet
Startup args:
--snap.stop --snapshots=true --authrpc.jwtsecret /etc/ethereum/jwt.hex --authrpc.port 8553 --datadir /erigon --http=false --maxpeers 20 --ethash.dagdir /erigon/erigon-ethash --port 30305 --trustedpeers=enode://xxx
Rpc daemon "newBlockHeaders" subscription is failing to report new blocks on arrival, falling behind the chain, and seems to report multiple blocks together with a delay.
I wrote a simple script that subscribes for "newBlockHeaders" on both geth and erigon and reports on arrival and here's the result:
18:01:36.554 GETH block 15590537
18:01:48.416 GETH block 15590538
18:01:48.988 ERIGON block: 15590537
18:01:48.988 ERIGON block: 15590538
18:02:00.751 GETH block 15590539
18:02:12.385 GETH block 15590540
18:02:12.895 ERIGON block: 15590539
18:02:12.896 ERIGON block: 15590540
18:02:25.161 GETH block 15590541
18:02:36.083 GETH block 15590542
18:02:36.595 ERIGON block: 15590541
18:02:36.597 ERIGON block: 15590542
18:02:48.382 GETH block 15590543
18:02:48.716 ERIGON block: 15590543
18:03:01.038 GETH block 15590544
18:03:01.492 ERIGON block: 15590544
18:03:13.263 GETH block 15590545
18:03:13.649 ERIGON block: 15590545
18:03:24.796 GETH block 15590546
18:03:36.721 GETH block 15590547
18:03:37.077 ERIGON block: 15590546
18:03:37.079 ERIGON block: 15590547
18:03:49.207 GETH block 15590548
18:04:01.241 GETH block 15590549
18:04:12.434 GETH block 15590550
18:04:13.200 ERIGON block: 15590548
18:04:13.200 ERIGON block: 15590549
18:04:13.201 ERIGON block: 15590550
18:04:24.971 GETH block 15590551
18:04:25.357 ERIGON block: 15590551
18:04:37.168 GETH block 15590552
18:04:37.513 ERIGON block: 15590552
18:04:49.041 GETH block 15590553
18:04:49.361 ERIGON block: 15590553
Also if I simply call getBlockNumber() on erigon, when a new block on geth arrives (while also giving erigon 1 extra second to catch up), erigon is very often 1-2 blocks behind
In erigon logs, I've noticed the RPC daemon is beeing notifdied very often about the same block or about multiple blocks.
[INFO] [09-22|23:01:24.046] [5/7 Execution] Completed on block=15590536
[INFO] [09-22|23:01:24.063] Timings (slower than 50ms) Headers=11.182s
[INFO] [09-22|23:01:24.063] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:01:24.067] Commit cycle in=4.050624ms
[INFO] [09-22|23:01:24.067] RPC Daemon notified of new headers from=15590535 to=15590536 header sending=5.29µs log sending=210ns
[INFO] [09-22|23:01:24.067] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-22|23:01:24.134] Flushing in-memory state
[INFO] [09-22|23:01:24.163] Timings (slower than 50ms) Headers=80ms
[INFO] [09-22|23:01:24.163] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:01:24.314] Commit cycle in=150.982957ms
[INFO] [09-22|23:01:24.314] RPC Daemon notified of new headers from=15590535 to=15590536 header sending=10.69µs log sending=160ns
[INFO] [09-22|23:01:24.314] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-22|23:01:26.828] [txpool] stat block=15590536 pending=167 baseFee=30000 queued=29999 alloc=2.4GB sys=3.8GB
[INFO] [09-22|23:01:36.056] [5/7 Execution] Completed on block=15590537
[INFO] [09-22|23:01:36.253] Timings (slower than 50ms) Execution=135ms IntermediateHashes=185ms
[INFO] [09-22|23:01:36.253] Tables PlainState=80.0KB AccountChangeSet=24.0KB StorageChangeSet=64.0KB BlockTransaction=128.0KB TransactionLog=140.0KB FreeList=0B
[INFO] [09-22|23:01:36.254] Timings (slower than 50ms) Headers=11.939s
[INFO] [09-22|23:01:36.254] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:01:36.259] Commit cycle in=5.380113ms
[INFO] [09-22|23:01:36.259] RPC Daemon notified of new headers from=15590536 to=15590537 header sending=5.21µs log sending=150ns
[INFO] [09-22|23:01:36.260] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-22|23:01:47.890] [5/7 Execution] Completed on block=15590538
[INFO] [09-22|23:01:47.958] Timings (slower than 50ms) IntermediateHashes=63ms
[INFO] [09-22|23:01:47.958] Tables PlainState=152.0KB AccountChangeSet=40.0KB StorageChangeSet=80.0KB BlockTransaction=192.0KB TransactionLog=200.0KB FreeList=0B
[INFO] [09-22|23:01:47.958] Timings (slower than 50ms) Headers=11.698s
[INFO] [09-22|23:01:47.958] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:01:47.963] Commit cycle in=4.522396ms
[INFO] [09-22|23:01:47.963] RPC Daemon notified of new headers from=15590536 to=15590538 header sending=5.37µs log sending=151ns
[INFO] [09-22|23:01:47.963] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-22|23:01:47.963] Flushing in-memory state
[INFO] [09-22|23:01:48.391] Timings (slower than 50ms) Headers=207ms CallTraces=52ms StorageHistoryIndex=62ms
[INFO] [09-22|23:01:48.391] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:01:48.524] Commit cycle in=133.638276ms
[INFO] [09-22|23:01:48.524] RPC Daemon notified of new headers from=15590536 to=15590538 header sending=9.818µs log sending=441ns
[INFO] [09-22|23:01:48.525] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-22|23:02:00.254] [5/7 Execution] Completed on block=15590539
[INFO] [09-22|23:02:00.432] Timings (slower than 50ms) Execution=98ms IntermediateHashes=172ms
[INFO] [09-22|23:02:00.432] Tables PlainState=44.0KB AccountChangeSet=20.0KB StorageChangeSet=32.0KB BlockTransaction=96.0KB TransactionLog=64.0KB FreeList=0B
[INFO] [09-22|23:02:00.433] Timings (slower than 50ms) Headers=11.907s
[INFO] [09-22|23:02:00.433] Tables PlainState=74.3GB AccountChangeSet=163.9GB StorageChangeSet=242.3GB BlockTransaction=160.6GB TransactionLog=451.2GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-22|23:02:00.437] Commit cycle in=4.39784ms
[INFO] [09-22|23:02:00.437] RPC Daemon notified of new headers from=15590538 to=15590539 header sending=12.193µs log sending=340ns
can you try this branch out flushing-blocks
can you try this branch out
flushing-blocks
Unfortunately no, since when I update https://github.com/ledgerwatch/erigon/issues/5482 reoccurs and erigon is not working
can you try this branch out
flushing-blocks
Looks like nothing changed with this branch.
20:46:12.000 GETH block 15598524
20:46:12.165 ERIGON block: 15598524
20:46:24.846 GETH block 15598525
20:46:37.556 GETH block 15598526
20:46:49.006 GETH block 15598527
20:46:49.748 ERIGON block: 15598525
20:46:49.749 ERIGON block: 15598526
20:46:49.749 ERIGON block: 15598527
20:47:00.555 GETH block 15598528
20:47:00.779 ERIGON block: 15598528
20:47:12.589 GETH block 15598529
20:47:12.910 ERIGON block: 15598529
20:47:25.004 GETH block 15598530
20:47:36.098 GETH block 15598531
20:47:36.597 ERIGON block: 15598530
20:47:36.598 ERIGON block: 15598531
20:47:48.534 GETH block 15598532
20:47:48.938 ERIGON block: 15598532
20:48:00.301 GETH block 15598533
20:48:00.650 ERIGON block: 15598533
20:48:11.777 GETH block 15598534
20:48:11.843 ERIGON block: 15598534
20:48:24.140 GETH block 15598535
20:48:36.513 GETH block 15598536
20:48:37.121 ERIGON block: 15598535
20:48:37.121 ERIGON block: 15598536
20:48:48.519 GETH block 15598537
20:48:48.707 ERIGON block: 15598537
20:49:03.071 GETH block 15598538
20:49:12.626 GETH block 15598539
20:49:13.158 ERIGON block: 15598538
20:49:13.159 ERIGON block: 15598539
20:49:25.343 GETH block 15598540
20:49:25.606 ERIGON block: 15598540
20:49:36.484 GETH block 15598541
I want to see the rpc notification. If the rpc is being notified of the same block multiple times still.
times
Looks like sometimes it does, although less than before, and some commit times have gone up quite a bit
[INFO] [09-24|01:53:39.650] [p2p] GoodPeers eth66=11
[INFO] [09-24|01:53:40.001] [txpool] stat block=15598561 pending=128 baseFee=30000 queued=29993 alloc=2.7GB sys=3.2GB
[INFO] [09-24|01:53:48.912] [5/7 Execution] Completed on block=15598562
[INFO] [09-24|01:53:49.016] Timings (slower than 50ms) Execution=76ms IntermediateHashes=98ms
[INFO] [09-24|01:53:49.016] Tables PlainState=52.0KB AccountChangeSet=24.0KB StorageChangeSet=36.0KB BlockTransaction=100.0KB TransactionLog=68.0KB FreeList=0B
[INFO] [09-24|01:53:49.017] Timings (slower than 50ms) Headers=12.186s
[INFO] [09-24|01:53:49.017] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.5MB ReclaimableSpace=10.5GB
[INFO] [09-24|01:53:49.022] Commit cycle in=4.521241ms
[INFO] [09-24|01:53:49.022] RPC Daemon notified of new headers from=15598561 to=15598562 header sending=5.431µs log sending=170ns
[INFO] [09-24|01:53:49.022] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:53:49.026] Flushing in-memory state
[INFO] [09-24|01:53:49.372] Commit cycle in=209.636705ms
[INFO] [09-24|01:53:49.373] RPC Daemon notified of new headers from=15598561 to=15598562 header sending=11.021µs log sending=181ns
[INFO] [09-24|01:53:49.373] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:53:59.583] [5/7 Execution] Completed on block=15598563
[INFO] [09-24|01:53:59.617] Timings (slower than 50ms) Headers=10.244s
[INFO] [09-24|01:53:59.618] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:53:59.622] Commit cycle in=4.411871ms
[INFO] [09-24|01:53:59.622] RPC Daemon notified of new headers from=15598562 to=15598563 header sending=5.581µs log sending=181ns
[INFO] [09-24|01:53:59.622] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:53:59.687] Flushing in-memory state
[INFO] [09-24|01:53:59.736] Timings (slower than 50ms) Headers=81ms
[INFO] [09-24|01:53:59.736] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:53:59.865] Commit cycle in=129.728155ms
[INFO] [09-24|01:53:59.866] RPC Daemon notified of new headers from=15598562 to=15598563 header sending=14.177µs log sending=170ns
[INFO] [09-24|01:53:59.866] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:12.250] [5/7 Execution] Completed on block=15598564
[INFO] [09-24|01:54:12.464] Timings (slower than 50ms) Execution=161ms IntermediateHashes=202ms
[INFO] [09-24|01:54:12.464] Tables PlainState=88.0KB AccountChangeSet=32.0KB StorageChangeSet=60.0KB BlockTransaction=192.0KB TransactionLog=180.0KB FreeList=0B
[INFO] [09-24|01:54:12.465] Timings (slower than 50ms) Headers=12.599s
[INFO] [09-24|01:54:12.465] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:12.470] Commit cycle in=4.449303ms
[INFO] [09-24|01:54:12.470] RPC Daemon notified of new headers from=15598563 to=15598564 header sending=5.48µs log sending=150ns
[INFO] [09-24|01:54:12.470] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:24.453] [5/7 Execution] Completed on block=15598565
[INFO] [09-24|01:54:24.455] Timings (slower than 50ms) Headers=11.984s
[INFO] [09-24|01:54:24.455] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:24.459] Commit cycle in=4.449391ms
[INFO] [09-24|01:54:24.460] RPC Daemon notified of new headers from=15598563 to=15598565 header sending=7.424µs log sending=210ns
[INFO] [09-24|01:54:24.460] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:24.547] Flushing in-memory state
[INFO] [09-24|01:54:24.922] Timings (slower than 50ms) Headers=265ms StorageHistoryIndex=53ms
[INFO] [09-24|01:54:24.922] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:25.112] Commit cycle in=189.75453ms
[INFO] [09-24|01:54:25.112] RPC Daemon notified of new headers from=15598563 to=15598565 header sending=11.402µs log sending=170ns
[INFO] [09-24|01:54:25.112] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:37.209] [5/7 Execution] Completed on block=15598566
[INFO] [09-24|01:54:37.421] Timings (slower than 50ms) Execution=146ms IntermediateHashes=201ms
[INFO] [09-24|01:54:37.421] Tables PlainState=92.0KB AccountChangeSet=32.0KB StorageChangeSet=72.0KB BlockTransaction=188.0KB TransactionLog=160.0KB FreeList=0B
[INFO] [09-24|01:54:37.423] Timings (slower than 50ms) Headers=12.31s
[INFO] [09-24|01:54:37.423] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:37.428] Commit cycle in=4.770021ms
[INFO] [09-24|01:54:37.428] RPC Daemon notified of new headers from=15598565 to=15598566 header sending=5.179µs log sending=150ns
[INFO] [09-24|01:54:37.428] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:39.994] [txpool] stat block=15598566 pending=110 baseFee=30000 queued=30000 alloc=2.3GB sys=3.2GB
[INFO] [09-24|01:54:48.103] [5/7 Execution] Completed on block=15598567
[INFO] [09-24|01:54:48.210] Timings (slower than 50ms) Execution=111ms IntermediateHashes=100ms
[INFO] [09-24|01:54:48.210] Tables PlainState=180.0KB AccountChangeSet=48.0KB StorageChangeSet=100.0KB BlockTransaction=260.0KB TransactionLog=232.0KB FreeList=0B
[INFO] [09-24|01:54:48.210] Timings (slower than 50ms) Headers=10.782s
[INFO] [09-24|01:54:48.210] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:48.215] Commit cycle in=4.76507ms
[INFO] [09-24|01:54:48.215] RPC Daemon notified of new headers from=15598565 to=15598567 header sending=9.167µs log sending=221ns
[INFO] [09-24|01:54:48.215] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:54:48.219] Flushing in-memory state
[INFO] [09-24|01:54:48.631] Timings (slower than 50ms) Headers=163ms CallTraces=55ms StorageHistoryIndex=80ms
[INFO] [09-24|01:54:48.631] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:54:48.966] Commit cycle in=334.795077ms
[INFO] [09-24|01:54:48.966] RPC Daemon notified of new headers from=15598565 to=15598567 header sending=14.708µs log sending=170ns
[INFO] [09-24|01:54:48.966] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:55:00.119] [5/7 Execution] Completed on block=15598568
[INFO] [09-24|01:55:00.257] Timings (slower than 50ms) Execution=89ms IntermediateHashes=132ms
[INFO] [09-24|01:55:00.257] Tables PlainState=48.0KB AccountChangeSet=24.0KB StorageChangeSet=40.0KB BlockTransaction=96.0KB TransactionLog=80.0KB FreeList=0B
[INFO] [09-24|01:55:00.258] Timings (slower than 50ms) Headers=11.291s
[INFO] [09-24|01:55:00.258] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:55:00.263] Commit cycle in=4.743307ms
[INFO] [09-24|01:55:00.263] RPC Daemon notified of new headers from=15598567 to=15598568 header sending=6.793µs log sending=201ns
[INFO] [09-24|01:55:00.263] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-24|01:55:00.266] Flushing in-memory state
[INFO] [09-24|01:55:00.505] Timings (slower than 50ms) Headers=140ms
[INFO] [09-24|01:55:00.505] Tables PlainState=74.4GB AccountChangeSet=164.0GB StorageChangeSet=242.5GB BlockTransaction=161.3GB TransactionLog=451.8GB FreeList=10.4MB ReclaimableSpace=10.4GB
[INFO] [09-24|01:55:00.594] Commit cycle in=88.844585ms
[INFO] [09-24|01:55:00.594] RPC Daemon notified of new headers from=15598567 to=15598568 header sending=14.288µs log sending=240ns
It looks like on erigon 93add6cacf151fbaa5eff9a0d7854e86b64ff371 logs the blocks come in correctly, but every few blocks erigon "forgets" to notify the RPC Daemon, and all the calls point to the last notified block.
[INFO] [09-29|23:48:02.221] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:48:11.879] [2/16 Headers] Handling new payload height=15640884 hash=0xe01654d8712ba05125d7e89490361585ce0d772409b0486cbe47071004216a94
[INFO] [09-29|23:48:12.046] Commit cycle in=3.946015ms
[INFO] [09-29|23:48:12.046] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:48:12.047] [2/16 Headers] Fork choice update: flushing in-memory state (built by previous newPayload)
[INFO] [09-29|23:48:12.390] Commit cycle in=174.627347ms
[INFO] [09-29|23:48:12.390] Timings (slower than 50ms) Headers=82ms
[INFO] [09-29|23:48:12.390] Tables PlainState=74.9GB AccountChangeSet=164.7GB StorageChangeSet=243.8GB BlockTransaction=164.9GB TransactionLog=455.2GB FreeList=10.8MB ReclaimableSpace=10.8GB
[INFO] [09-29|23:48:12.390] RPC Daemon notified of new headers from=15640883 to=15640884 hash=0xe01654d8712ba05125d7e89490361585ce0d772409b0486cbe47071004216a94 header sending=9.959µs log sending=240ns
[INFO] [09-29|23:48:12.390] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:48:18.102] [p2p] GoodPeers eth66=20
[INFO] [09-29|23:48:19.237] [txpool] stat block=15640884 pending=10000 baseFee=0 queued=29963 alloc=4.3GB sys=5.6GB
[INFO] [09-29|23:48:23.886] [2/16 Headers] Handling new payload height=15640885 hash=0x6983c5643f15e8fc77526eed3a969a97e3bbe18b6cb4c3cb302bff4a2ec56cea
[INFO] [09-29|23:48:24.205] Commit cycle in=4.14911ms
[INFO] [09-29|23:48:24.205] [2/16 Headers] Waiting for Consensus Layer...
[WARN] [09-29|23:48:35.173] Served conn=127.0.0.1:46184 method=engine_exchangeTransitionConfigurationV1 reqid=1 t=58.2µs err="the execution layer doesn't have a terminal total difficulty. expected: 0xc70d808a128d7380000"
[INFO] [09-29|23:48:35.848] [2/16 Headers] Handling new payload height=15640886 hash=0x0942496cfb6f09b1070dec0b4f7c66a129fdfc0663312a4b84e5fc20d39bd307
[INFO] [09-29|23:48:36.332] Commit cycle in=4.070661ms
[INFO] [09-29|23:48:36.332] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:48:48.224] [2/16 Headers] Handling new payload height=15640887 hash=0xad3f6982482caaec03180586399da16c973584cbe2a6c64b315c6a816bff9a2f
[INFO] [09-29|23:48:48.811] Commit cycle in=4.083537ms
[INFO] [09-29|23:48:48.812] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:48:59.805] [2/16 Headers] Handling new payload height=15640888 hash=0xa50152809f2c2fe16f2f164cbdbb1b72de51fedfdc0a96a7e013d5ee46dd4175
[INFO] [09-29|23:49:00.031] Commit cycle in=4.105569ms
[INFO] [09-29|23:49:00.031] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:49:00.032] [2/16 Headers] Fork choice update: flushing in-memory state (built by previous newPayload)
[INFO] [09-29|23:49:01.342] Commit cycle in=317.670225ms
[INFO] [09-29|23:49:01.342] Timings (slower than 50ms) Headers=272ms CallTraces=174ms AccountHistoryIndex=126ms StorageHistoryIndex=179ms LogIndex=116ms TxLookup=123ms
[INFO] [09-29|23:49:01.342] Tables PlainState=74.9GB AccountChangeSet=164.7GB StorageChangeSet=243.8GB BlockTransaction=164.9GB TransactionLog=455.2GB FreeList=10.8MB ReclaimableSpace=10.8GB
[INFO] [09-29|23:49:01.342] RPC Daemon notified of new headers from=15640884 to=15640888 hash=0xa50152809f2c2fe16f2f164cbdbb1b72de51fedfdc0a96a7e013d5ee46dd4175 header sending=12.985µs log sending=311ns
[INFO] [09-29|23:49:01.342] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:49:12.502] [2/16 Headers] Handling new payload height=15640889 hash=0xeeaa57caca5c7481b26cd0a16a9b0ea92d774487cefe4d7ebfecba60086d04cd
[INFO] [09-29|23:49:12.888] Commit cycle in=4.098637ms
[INFO] [09-29|23:49:12.888] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:49:18.458] [txpool] stat block=15640889 pending=10000 baseFee=0 queued=29991 alloc=4.7GB sys=5.6GB
[INFO] [09-29|23:49:24.457] [2/16 Headers] Handling new payload height=15640890 hash=0xf80eb516592d2fbd08a13ec11a93f77e0ba381d2173fa715e847068b31a43848
[INFO] [09-29|23:49:24.771] Commit cycle in=4.114456ms
[INFO] [09-29|23:49:24.771] [2/16 Headers] Waiting for Consensus Layer...
[WARN] [09-29|23:49:35.174] Served conn=127.0.0.1:43852 method=engine_exchangeTransitionConfigurationV1 reqid=1 t=67.077µs err="the execution layer doesn't have a terminal total difficulty. expected: 0xc70d808a128d7380000"
[INFO] [09-29|23:49:36.030] [2/16 Headers] Handling new payload height=15640891 hash=0x947565c88b6a3306755c49753672b3749146837e94d91e1474b7c1b78438e527
[INFO] [09-29|23:49:36.265] Commit cycle in=3.977097ms
[INFO] [09-29|23:49:36.265] [2/16 Headers] Waiting for Consensus Layer...
[INFO] [09-29|23:49:36.267] [2/16 Headers] Fork choice update: flushing in-memory state (built by previous newPayload)
[INFO] [09-29|23:49:37.078] Commit cycle in=224.298511ms
[INFO] [09-29|23:49:37.078] Timings (slower than 50ms) Headers=177ms CallTraces=94ms AccountHistoryIndex=81ms StorageHistoryIndex=113ms LogIndex=61ms TxLookup=59ms
[INFO] [09-29|23:49:37.078] Tables PlainState=74.9GB AccountChangeSet=164.7GB StorageChangeSet=243.8GB BlockTransaction=164.9GB TransactionLog=455.2GB FreeList=10.8MB ReclaimableSpace=10.8GB
[INFO] [09-29|23:49:37.078] RPC Daemon notified of new headers from=15640888 to=15640891 hash=0x947565c88b6a3306755c49753672b3749146837e94d91e1474b7c1b78438e527 header sending=15.299µs log sending=792ns
[INFO] [09-29|23:49:37.078] [2/16 Headers] Waiting for Consensus Layer...
This issue is stale because it has been open for 40 days with no activity. Remove stale label or comment, or this will be closed in 7 days.
This issue was closed because it has been stalled for 7 days with no activity.