Can't sync from scratch with --externalcl

Open 86b opened this issue 1 year ago • 9 comments

Can't sync from scratch on numerous ETH networks (mainnet, sepolia, holesky) on v3.0.0-alpha4 with --externalcl declared (in this case when using Lighthouse). The following error will eventually show with a varying block number depending on when you started syncing from scratch:

[EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20833531"

Oct 04 '24 18:10 86b

erigon.service:

[Unit]
Description=Erigon
After=network.target
StartLimitIntervalSec=10

[Service]
Type=simple
Restart=always
RestartSec=10
User=ubuntu
ExecStart=/data/erigon/build/bin/erigon \
  --datadir=/data/.erigon \
  --chain=holesky \
  --externalcl \
  --db.pagesize=4kb \
  --db.size.limit=8TB \
  --downloader.disable.ipv6=true \
  --http.addr=0.0.0.0 \
  --http.port=8545 \
  --http.api=net,web3,eth,admin,debug,txpool,engine,rpc,trace \
  --ws \
  --metrics \
  --metrics.addr=0.0.0.0 \
  --metrics.port=6060 \
  --authrpc.port=8551 \
  --authrpc.addr=0.0.0.0 \
  --authrpc.jwtsecret=/data/jwt/jwtsecret \
  --authrpc.vhosts=* \
  --http.vhosts=* \
  --http.corsdomain=* \
  --torrent.download.slots=10 \
  --torrent.download.rate=2500mb \
  --rpc.gascap=5000000000 \
  --log.console.verbosity=dbug

TimeoutStopSec=240

[Install]
WantedBy=multi-user.target

lighthouse.service:

[Unit]
Description=Lighthouse
After=network.target
StartLimitIntervalSec=10

[Service]
Type=simple
Restart=always
RestartSec=10
User=ubuntu
ExecStart=/data/lighthouse/target/release/lighthouse bn \
 --datadir /data/.lighthouse \
 --listen-address 0.0.0.0 \
 --disable-upnp \
 --network holesky \
 --execution-endpoint http://127.0.0.1:8551 \
 --execution-jwt /data/jwt/jwtsecret \
 --http \
 --http-address 0.0.0.0 \
 --metrics \
 --metrics-address 0.0.0.0 \
 --prune-blobs=false \
 --disable-backfill-rate-limiting \
 --disable-inbound-rate-limiter \
 --genesis-backfill \
 --target-peers=200 \
 --checkpoint-sync-url https://beaconstate-holesky.chainsafe.io

TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

Oct 04 '24 18:10 86b

Could be related to https://github.com/erigontech/erigon/issues/11509

Oct 08 '24 10:10 somnergy

Please attach complete logs @86b

Oct 08 '24 10:10 somnergy

Not OP but get same issue, logs repeat after a while:

version: v3.0.0-alpha4

[INFO] [10-08|21:31:39.248] [1/6 OtterSync] Downloading complete     time=19.881217863s
[INFO] [10-08|21:32:21.277] [1/6 OtterSync] Requesting downloads
[INFO] [10-08|21:32:21.890] [snapshots:download] Stat                blocks=20.87M indices=20.87M alloc=6.8GB sys=9.4GB
[INFO] [10-08|21:32:21.892] [snapshots:history] Stat                 blocks=20.87M txs=2.57B txNum2blockNum="1024=14760K,1536=19814K,1600=20426K,1632=20746K,1640=20825K,1644=20864K,1645=20872K" first_history_idx_in_db=20872771 last_comitment_block=20872771 last_comitment_tx_num=2570312500 alloc=6.8GB sys=9.4GB
[INFO] [10-08|21:32:21.892] [1/6 OtterSync] DONE                     in=1m2.525646653s block=20877999
[INFO] [10-08|21:32:25.029] [NewPayload] Handling new payload        height=20923569 hash=0xb0ce193d367a354c01277960fa10cb44c2b2655f7d74defca36b8035e3937b00
[INFO] [10-08|21:32:25.029] [EngineBlockDownloader] Downloading PoS headers... hash=0x167914e823f330aaefca2b3ceeb2235270072f8296f6699c9393d2d1dec930ed requestId=0
[INFO] [10-08|21:32:42.528] [snapshots:blocks:retire] Stat           blocks=20.87M indices=20.87M alloc=6.0GB sys=9.4GB
[INFO] [10-08|21:32:55.125] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:33:21.428] [downloader] Downloaded PoS Headers      now=20904752 blk/sec=262.400
[INFO] [10-08|21:33:27.224] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:33:51.624] [downloader] Downloaded PoS Headers      now=20902448 blk/sec=76.800
[INFO] [10-08|21:33:58.025] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:34:19.323] [p2p] GoodPeers                          eth68=1
[INFO] [10-08|21:34:19.430] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=6.3GB sys=11.3GB
[INFO] [10-08|21:34:20.829] [downloader] Downloaded PoS Headers      now=20899376 blk/sec=102.400
[INFO] [10-08|21:34:25.123] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:34:25.724] [mem] memory stats                       Rss=12.7GB Size=0B Pss=12.7GB SharedClean=1.3MB SharedDirty=0B PrivateClean=4.2GB PrivateDirty=8.4GB Referenced=12.7GB Anonymous=8.4GB Swap=0B alloc=6.4GB sys=11.4GB
[INFO] [10-08|21:34:49.523] [downloader] Downloaded PoS Headers      now=20895152 blk/sec=140.800
[INFO] [10-08|21:34:55.029] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:35:19.628] [downloader] Downloaded PoS Headers      now=20888048 blk/sec=236.800
[INFO] [10-08|21:35:25.523] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:35:37.829] [NewPayload] Handling new payload        height=20923585 hash=0x5ca2a651587f001365c784f78f0eff5530770d05712ef3d246bad7643dc79964
[WARN] [10-08|21:35:43.928] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:35:49.026] [NewPayload] Handling new payload        height=20923586 hash=0xff65e208247e7811cc2102da908fe925fcfd3626b9581662c94b075c3f7821e0
[INFO] [10-08|21:35:49.026] [EngineBlockDownloader] Downloading PoS headers... hash=0x5ca2a651587f001365c784f78f0eff5530770d05712ef3d246bad7643dc79964 requestId=0
[INFO] [10-08|21:36:19.127] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:36:47.628] [engine] CurrentHeader                   err="ethereumExecutionModule.CurrentHeader: could not begin database tx context canceled"
[INFO] [10-08|21:36:53.427] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:36:53.427] [downloader] Downloaded PoS Headers      now=20899201 blk/sec=204.800
[INFO] [10-08|21:37:19.323] [p2p] GoodPeers                          eth68=2
[INFO] [10-08|21:37:19.325] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:37:19.425] [downloader] Downloaded PoS Headers      now=20894593 blk/sec=153.600
[INFO] [10-08|21:37:19.427] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.9GB sys=12.5GB
[INFO] [10-08|21:37:21.524] [mem] memory stats                       Rss=13.1GB Size=0B Pss=13.1GB SharedClean=1.3MB SharedDirty=0B PrivateClean=4.2GB PrivateDirty=8.8GB Referenced=13.1GB Anonymous=8.8GB Swap=0B alloc=6.0GB sys=12.5GB
[INFO] [10-08|21:37:49.226] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:37:49.629] [downloader] Downloaded PoS Headers      now=20883457 blk/sec=371.200
[INFO] [10-08|21:38:01.025] [NewPayload] Handling new payload        height=20923597 hash=0xb6da35cdf0a311c599c9ee54d793fa6dfb058a07b547b5faf91e010e40c8201b
[WARN] [10-08|21:38:08.030] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:38:13.026] [NewPayload] Handling new payload        height=20923598 hash=0x7d9d02c3b3fc48905a76d23510a6509ef8923a0acb66eef63d89f6b366290f7a
[INFO] [10-08|21:38:13.026] [EngineBlockDownloader] Downloading PoS headers... hash=0xb6da35cdf0a311c599c9ee54d793fa6dfb058a07b547b5faf91e010e40c8201b requestId=0
[INFO] [10-08|21:38:43.328] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:38:49.423] [downloader] Downloaded PoS Headers      now=20909965 blk/sec=384.000
[INFO] [10-08|21:39:13.123] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:39:19.623] [downloader] Downloaded PoS Headers      now=20898445 blk/sec=384.000
[INFO] [10-08|21:39:43.123] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:39:49.423] [downloader] Downloaded PoS Headers      now=20887117 blk/sec=377.600
[INFO] [10-08|21:40:00.828] [NewPayload] Handling new payload        height=20923607 hash=0x51243de9553241904d85e17f6fbd6b5bd153d1734885febed73a28010ff360ae
[INFO] [10-08|21:40:12.628] [NewPayload] Handling new payload        height=20923608 hash=0x9ad304c80636995807ea914f3bdd0cac9d2fcbfcbe436c719571ae100467839f
[WARN] [10-08|21:40:15.026] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:40:19.323] [p2p] GoodPeers                          eth68=4 eth67=1
[INFO] [10-08|21:40:19.424] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.1GB sys=12.5GB
[INFO] [10-08|21:40:21.324] [mem] memory stats                       Rss=11.6GB Size=0B Pss=11.6GB SharedClean=1.3MB SharedDirty=0B PrivateClean=4.2GB PrivateDirty=7.3GB Referenced=11.6GB Anonymous=7.3GB Swap=0B alloc=5.1GB sys=12.5GB
[INFO] [10-08|21:40:24.725] [NewPayload] Handling new payload        height=20923609 hash=0xe7139e0f28a00f9999fab8bf1916639d9aa1d3d9b2d67a985e7b64c5706ecde5
[INFO] [10-08|21:40:24.726] [EngineBlockDownloader] Downloading PoS headers... hash=0x9ad304c80636995807ea914f3bdd0cac9d2fcbfcbe436c719571ae100467839f requestId=0
[INFO] [10-08|21:40:54.930] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:41:19.434] [downloader] Downloaded PoS Headers      now=20900568 blk/sec=396.800
[INFO] [10-08|21:41:25.025] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:41:49.430] [downloader] Downloaded PoS Headers      now=20888664 blk/sec=396.800
[INFO] [10-08|21:41:54.829] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:42:12.828] [NewPayload] Handling new payload        height=20923618 hash=0xbb588199fa09d74583491741eda316f2865cef2198fa054b13ee6fca36e181f2
[WARN] [10-08|21:42:18.827] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:42:25.424] [NewPayload] Handling new payload        height=20923619 hash=0xcc0424b4cc27bf853b72211a6c373a85982c4a5bd1802ac98ebcfc24e98d1639
[INFO] [10-08|21:42:25.424] [EngineBlockDownloader] Downloading PoS headers... hash=0xbb588199fa09d74583491741eda316f2865cef2198fa054b13ee6fca36e181f2 requestId=0
[INFO] [10-08|21:42:55.424] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:43:05.128] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:43:07.642] [snapshots:merge] Stat                   blocks=20.87M indices=20.87M alloc=7.6GB sys=12.5GB
[INFO] [10-08|21:43:12.924] [NewPayload] Handling new payload        height=20923623 hash=0x41dc30498b9a87481b24ce22d3389b7fbeb0df42dedbb7c68891b8f0afd2809e
[INFO] [10-08|21:43:12.925] [EngineBlockDownloader] Downloading PoS headers... hash=0xed95dda89970dff7c12b2971648963b9800c60fabcab99fa86694f1b58c41440 requestId=0
[INFO] [10-08|21:43:19.249] [p2p] GoodPeers                          eth68=3
[INFO] [10-08|21:43:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=4.7GB sys=12.5GB
[INFO] [10-08|21:43:19.726] [mem] memory stats                       Rss=9.3GB Size=0B Pss=9.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=5.6GB Referenced=9.3GB Anonymous=5.6GB Swap=0B alloc=4.7GB sys=12.5GB
[INFO] [10-08|21:43:29.378] [snapshots:blocks:retire] Stat           blocks=20.87M indices=20.87M alloc=6.1GB sys=12.5GB
[INFO] [10-08|21:43:42.925] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:43:49.346] [downloader] Downloaded PoS Headers      now=20889062 blk/sec=1056.000
[WARN] [10-08|21:43:55.253] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:44:00.556] [NewPayload] Handling new payload        height=20923627 hash=0x20694f472da36f68bbf79e4f4728466b712f3b9ac7abc9209a6b87623b1e1cb1
[INFO] [10-08|21:44:00.556] [EngineBlockDownloader] Downloading PoS headers... hash=0x5ffe6a555d7ef0cfaa2960f773cae9e36e934606ca2aa804de0df306c9f997d6 requestId=0
[INFO] [10-08|21:44:30.556] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:44:34.449] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:44:36.724] [NewPayload] Handling new payload        height=20923630 hash=0xce96b99f11a8917e7b9bda96747e7df5b97e053d83fb69a5c6cba6450fdd4fbe
[INFO] [10-08|21:44:36.724] [EngineBlockDownloader] Downloading PoS headers... hash=0x552efdb90087442b0ac4cac0acb14383e0afe975a64a3580264ebfb5db538637 requestId=0
[INFO] [10-08|21:45:06.724] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:45:11.826] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:45:14.311] [NewPayload] Handling new payload        height=20923633 hash=0x22083ba9c7508ed36c9b51b44dc179768e1271003d5ddc07aee1882e7814ebc3
[INFO] [10-08|21:45:14.312] [EngineBlockDownloader] Downloading PoS headers... hash=0x54cdc96748ab7b5d27981f9d80ae5f236dabb5bb1abde74d0046ac8dd51028d0 requestId=0
[WARN] [10-08|21:45:42.218] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:45:49.052] [NewPayload] Handling new payload        height=20923636 hash=0x0eaf64f15f4dfc2d61699ada4a26db97657b74739050acb8046d0a010a3a2255
[INFO] [10-08|21:45:49.052] [EngineBlockDownloader] Downloading PoS headers... hash=0x46296f67775c115a586906d2145025c7144dfa2b792b355eba4e12d21674adfe requestId=0
[INFO] [10-08|21:46:19.052] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:46:19.249] [p2p] GoodPeers                          eth68=5 eth67=1
[INFO] [10-08|21:46:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=6.5GB sys=12.5GB
[INFO] [10-08|21:46:19.356] [mem] memory stats                       Rss=13.2GB Size=0B Pss=13.2GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.5GB Referenced=13.2GB Anonymous=9.5GB Swap=0B alloc=6.5GB sys=12.5GB
[WARN] [10-08|21:46:19.863] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:46:24.859] [NewPayload] Handling new payload        height=20923639 hash=0x43f302c898a88a102db98dfe52934da4628fd3c57d396a6553cfe71f03134ee0
[INFO] [10-08|21:46:24.859] [EngineBlockDownloader] Downloading PoS headers... hash=0x818c39b9e16bc9560401c6e8a0c87e43ed69e926f2156a64b606ec1c666fe52b requestId=0
[WARN] [10-08|21:46:55.190] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:47:01.789] [NewPayload] Handling new payload        height=20923642 hash=0xa6030e69da304129613da34d72a039ed1963668f7db684a3f841a13ff1c16fb5
[INFO] [10-08|21:47:01.789] [EngineBlockDownloader] Downloading PoS headers... hash=0x566d5ca6ee18b0745258dda8158ca2de228a5f8c8a42e610ff8b11bd616b5030 requestId=0
[INFO] [10-08|21:47:31.789] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:47:32.667] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:47:36.818] [NewPayload] Handling new payload        height=20923645 hash=0xbc0ce55c6dfd520dab2625edf565ed18929f8b5c3cadc53bb6348aafe8d23b70
[INFO] [10-08|21:47:36.818] [EngineBlockDownloader] Downloading PoS headers... hash=0x66ba1d0a7281a11db3290b7fbf876d96945355e731280d10a5f39c191035b51e requestId=0
[WARN] [10-08|21:48:01.695] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:48:13.005] [NewPayload] Handling new payload        height=20923648 hash=0x19b63f42c47beb41a287c034dcad57c5cb5204772acaa9914ac1e9ac47ad1899
[INFO] [10-08|21:48:13.005] [EngineBlockDownloader] Downloading PoS headers... hash=0x26b336764b1f50c9aeda125afc0efacac81896dc09431297a3b5f307128b4093 requestId=0
[WARN] [10-08|21:48:42.805] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:48:48.637] [NewPayload] Handling new payload        height=20923651 hash=0x7f149ae09266e8a26829ab3465fbfb91360d11ab09a4f5c555be21510754e4c7
[INFO] [10-08|21:48:48.637] [EngineBlockDownloader] Downloading PoS headers... hash=0xe434b375bf3fc25c414bb06bd01e296f53354114c2bb8a527fbd55b75a5540ba requestId=0
[INFO] [10-08|21:49:18.638] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:49:19.249] [p2p] GoodPeers                          eth68=5 eth67=1
[INFO] [10-08|21:49:19.423] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.3GB sys=12.5GB
[INFO] [10-08|21:49:19.424] [downloader] Downloaded PoS Headers      now=20883906 blk/sec=1318.400
[INFO] [10-08|21:49:19.925] [mem] memory stats                       Rss=13.0GB Size=0B Pss=13.0GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.3GB Referenced=13.0GB Anonymous=9.3GB Swap=0B alloc=8.4GB sys=12.5GB
[WARN] [10-08|21:49:21.915] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:49:24.572] [NewPayload] Handling new payload        height=20923654 hash=0x96e99c75964f08df940d85ab3aeb82bd2ca983b302be8e95f024cf19ead9ad5a
[INFO] [10-08|21:49:24.572] [EngineBlockDownloader] Downloading PoS headers... hash=0x118e23421f465b4d440538689dc298fcd07d7fffc00c83755fc7cf4174fb1bd7 requestId=0
[WARN] [10-08|21:49:54.171] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:50:01.128] [NewPayload] Handling new payload        height=20923657 hash=0x89dc26df7de35f6f59587a63b7024961f13bd10370f89478f1aab46c3dc7079a
[INFO] [10-08|21:50:01.128] [EngineBlockDownloader] Downloading PoS headers... hash=0xfb1e920d0070c70ead3e831ac975e11a54c65e453924ee46b2a24d994f7912e4 requestId=0
[WARN] [10-08|21:50:28.855] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:50:36.709] [NewPayload] Handling new payload        height=20923660 hash=0xa4dffdc3f407bae49a37fa793fef965b91e4fbaec19129790622c8ef86e1672a
[INFO] [10-08|21:50:36.709] [EngineBlockDownloader] Downloading PoS headers... hash=0x7dd7d35d428690838526043786192252d13a071780ed4a968f97db2d8354dea4 requestId=0
[INFO] [10-08|21:51:06.710] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:51:15.756] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:51:24.592] [NewPayload] Handling new payload        height=20923664 hash=0x6b344c9832892daad795998b75264ba51ed7331c10c1a2083be150310434eb63
[INFO] [10-08|21:51:24.592] [EngineBlockDownloader] Downloading PoS headers... hash=0x83d7d5c2f0ed7fc2b038c503b00c68ad486080646967dd7f400a705d800c8e65 requestId=0
[INFO] [10-08|21:51:54.592] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:52:06.200] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:52:13.557] [NewPayload] Handling new payload        height=20923668 hash=0x1635c5716814ee8c6b924f7a5b3e2a85caf487deff224d902e02ed51d0eaf863
[INFO] [10-08|21:52:13.557] [EngineBlockDownloader] Downloading PoS headers... hash=0xf64599c612d2b7a4873be8e39c7509fc33a25aa651d33984e0b89304df65facb requestId=0
[INFO] [10-08|21:52:19.250] [p2p] GoodPeers                          eth68=5 eth67=1
[INFO] [10-08|21:52:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=6.1GB sys=12.5GB
[INFO] [10-08|21:52:19.346] [mem] memory stats                       Rss=13.0GB Size=0B Pss=13.0GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.3GB Referenced=13.0GB Anonymous=9.3GB Swap=0B alloc=6.1GB sys=12.5GB
[INFO] [10-08|21:52:43.558] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:52:47.160] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:52:48.835] [NewPayload] Handling new payload        height=20923671 hash=0x027b37aaea01644c4221a8f5d1e7d26d93f09453b6751714ec2aa1c625552cc7
[INFO] [10-08|21:52:48.835] [EngineBlockDownloader] Downloading PoS headers... hash=0xb57117b7b5e688219ece2704d71e56be996f14d9a73f20832c8385dc72d4d79a requestId=0
[WARN] [10-08|21:53:10.903] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:53:13.414] [NewPayload] Handling new payload        height=20923673 hash=0x53495ce31bf2968831efbe142c647fe4cf084dcaa1977a5aa03ebb2a2f7b9e45
[INFO] [10-08|21:53:13.414] [EngineBlockDownloader] Downloading PoS headers... hash=0xbb580a36aaad84752e48fc27b77a805bd578137dd50a16ebc50a4210bf77bfe9 requestId=0
[INFO] [10-08|21:53:43.414] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:53:49.099] [NewPayload] Handling new payload        height=20923676 hash=0x80db5f1274453badf30bba3f6e19bc7c5a987633dd4529e8436227cdc3f96ec9
[WARN] [10-08|21:53:49.449] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:54:00.975] [NewPayload] Handling new payload        height=20923677 hash=0xfde26161d6e7b5e7864b69429350ca9b418b1f87fb2d41c52ebef400615fb53c
[INFO] [10-08|21:54:00.975] [EngineBlockDownloader] Downloading PoS headers... hash=0x80db5f1274453badf30bba3f6e19bc7c5a987633dd4529e8436227cdc3f96ec9 requestId=0
[INFO] [10-08|21:54:30.975] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:54:38.585] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:54:49.039] [NewPayload] Handling new payload        height=20923681 hash=0x90b1de020932658e61d098bd2e6a040f2960208caf21d492612f14bf6bfb381c
[INFO] [10-08|21:54:49.039] [EngineBlockDownloader] Downloading PoS headers... hash=0xdc2ac56db049730e71d39b78ff3d802a2d6f9825bcc445856d9b41d63c9f7d2d requestId=0
[INFO] [10-08|21:55:19.039] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|21:55:19.249] [p2p] GoodPeers                          eth68=5
[INFO] [10-08|21:55:19.344] [mem] memory stats                       Rss=13.0GB Size=0B Pss=13.0GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.3GB Referenced=13.0GB Anonymous=9.3GB Swap=0B alloc=7.7GB sys=12.5GB
[INFO] [10-08|21:55:19.346] [downloader] Downloaded PoS Headers      now=20890272 blk/sec=1107.200
[INFO] [10-08|21:55:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.7GB sys=12.5GB
[WARN] [10-08|21:55:27.316] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:55:34.533] [EngineBlockDownloader] Downloading PoS headers... hash=0xc423cf9d0555ac9db80591abe105dccdaa06a7be7b24577d0014ed883bcd521a requestId=0
[INFO] [10-08|21:56:04.533] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:56:12.226] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:56:12.786] [NewPayload] Handling new payload        height=20923688 hash=0x54919999a74a61743b9e3bd32774a47e7de278ed207ce305d4086d4ed13f086b
[INFO] [10-08|21:56:12.786] [EngineBlockDownloader] Downloading PoS headers... hash=0xb3ab35b2ceffac5f0f6477274b5e78b222e9f7e2ab47c3d809ebb5d96794df49 requestId=0
[INFO] [10-08|21:56:42.786] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:56:46.490] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:56:48.765] [NewPayload] Handling new payload        height=20923691 hash=0x264411980150b6cd9954d5a765231d4a1a3b4dc5e20ef9f80e024640f0c8ba27
[INFO] [10-08|21:56:48.765] [EngineBlockDownloader] Downloading PoS headers... hash=0x12a33262046f11898179ebfb54f143a3b9c5f0ccd31c2efc724848fa549ac587 requestId=0
[WARN] [10-08|21:57:09.138] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:57:14.348] [NewPayload] Handling new payload        height=20923693 hash=0x885ded6eee9605cd818fc1cf902b8e56a4de4b89d5b2323b33755e413e019e2f
[INFO] [10-08|21:57:14.348] [EngineBlockDownloader] Downloading PoS headers... hash=0xc81a4f05ab3e75f11afa8dc2f309bc61f0968aab6addfc5c4ec5833873555528 requestId=0
[WARN] [10-08|21:57:35.102] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:57:38.378] [NewPayload] Handling new payload        height=20923695 hash=0x1852a16dabbb2e766820de0ee44d362c1bf36f9a8754421f2b4061ec15fe90bf
[INFO] [10-08|21:57:38.378] [EngineBlockDownloader] Downloading PoS headers... hash=0xe78645c9d47d4825ed29835ae6477dbd3473b120687a6276177f23dcd0e87f09 requestId=0
[WARN] [10-08|21:57:57.816] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:58:00.786] [NewPayload] Handling new payload        height=20923697 hash=0x08069bd5b50f4fc9e03ca68729163d5d3bb5109999d484527b64fd110db4e8a4
[INFO] [10-08|21:58:00.787] [EngineBlockDownloader] Downloading PoS headers... hash=0xd81c047f1f9d878c9a97f353d96f112b30e4cb38a199993318f5a3b2fd4412d2 requestId=0
[INFO] [10-08|21:58:19.249] [p2p] GoodPeers                          eth68=8 eth67=2
[INFO] [10-08|21:58:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.3GB sys=12.5GB
[INFO] [10-08|21:58:19.362] [mem] memory stats                       Rss=13.2GB Size=0B Pss=13.2GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.5GB Referenced=13.2GB Anonymous=9.5GB Swap=0B alloc=8.3GB sys=12.5GB
[WARN] [10-08|21:58:23.154] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:58:24.790] [NewPayload] Handling new payload        height=20923699 hash=0x2af376939553f7584a5a175a731752ef2a0c385b9fe45215e06eb219653cd5cf
[INFO] [10-08|21:58:24.790] [EngineBlockDownloader] Downloading PoS headers... hash=0xa58b4fc78ce3b917e34ab9d0c05962e170aa068c8f7482a1e3bac548deed2dca requestId=0
[WARN] [10-08|21:58:47.832] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:59:01.022] [NewPayload] Handling new payload        height=20923701 hash=0x45a221681e00367698e89cfacd28fb49dd47a65b474d63509fd8ce22aa47fdf2
[INFO] [10-08|21:59:01.022] [EngineBlockDownloader] Downloading PoS headers... hash=0xd45c4b358e4ac906c9e5b5edcec7900eb33389475c0b7ffc5416887e51647c72 requestId=0
[WARN] [10-08|21:59:19.039] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|21:59:25.295] [NewPayload] Handling new payload        height=20923703 hash=0xd068a1ae963f1829878670f61a95c637feb22ef784c7576be0ea01fbaee8b7a1
[INFO] [10-08|21:59:25.295] [EngineBlockDownloader] Downloading PoS headers... hash=0xc4f8a8e0bbd268584de056540ed822bcae2944b729e77eb36c7fded6b5911299 requestId=0
[INFO] [10-08|21:59:55.295] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|21:59:59.488] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:00:01.723] [NewPayload] Handling new payload        height=20923706 hash=0xf0666732f5542e4334095d4069bfbb88b76370eadc16cf149b100c7c02d2b242
[INFO] [10-08|22:00:01.723] [EngineBlockDownloader] Downloading PoS headers... hash=0x774d93c5137e17958eef3fe9a30dc889d2b7b761ce65a3dddd566ae739a6e8b0 requestId=0
[WARN] [10-08|22:00:29.909] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:00:37.514] [NewPayload] Handling new payload        height=20923709 hash=0x81602f81b216373cd73c0a5764d28a7e5fff4058655b23c45ab10ef579b5212f
[INFO] [10-08|22:00:37.514] [EngineBlockDownloader] Downloading PoS headers... hash=0x1f47eb44a477771cc71cd0c70f78198cb4829ad59dbbc13e904ac47928c03834 requestId=0
[WARN] [10-08|22:01:02.778] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:01:12.576] [NewPayload] Handling new payload        height=20923712 hash=0x4136fa25923c625bab2762ff9f3a48b0323a7c8a35652e4dceb5fff5fb62677e
[INFO] [10-08|22:01:12.576] [EngineBlockDownloader] Downloading PoS headers... hash=0x855cbaa3b6eeefac52ee67a044d7b5af09d57f169ba05efd7db7bd6fdbb1dab1 requestId=0
[INFO] [10-08|22:01:19.249] [p2p] GoodPeers                          eth67=3 eth68=9
[INFO] [10-08|22:01:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.1GB sys=12.5GB
[INFO] [10-08|22:01:19.363] [mem] memory stats                       Rss=13.2GB Size=0B Pss=13.2GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.6GB Referenced=13.2GB Anonymous=9.6GB Swap=0B alloc=8.1GB sys=12.5GB
[WARN] [10-08|22:01:39.943] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:01:48.671] [NewPayload] Handling new payload        height=20923715 hash=0xe632383df4f9371c5147bd639b049d09b6dc2aed3c532b31fd9b0ddb5a16e72d
[INFO] [10-08|22:01:48.672] [EngineBlockDownloader] Downloading PoS headers... hash=0xb86f73f49a6f6c2b478e376445b292589b9eee3bf54a8fd2743d8a0688647572 requestId=0
[INFO] [10-08|22:02:18.672] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:02:19.346] [downloader] Downloaded PoS Headers      now=20883586 blk/sec=1318.400
[WARN] [10-08|22:02:20.345] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:02:25.334] [NewPayload] Handling new payload        height=20923718 hash=0x769805ea8640f9b7899293f1c6b4902ecbb380171293d56a86fa25ef6c1d59a3
[INFO] [10-08|22:02:25.334] [EngineBlockDownloader] Downloading PoS headers... hash=0x1f654afb8a2bfd27016563a7641875c83962b1c2ede336a9e447c7d62a474bd2 requestId=0
[INFO] [10-08|22:02:55.335] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:03:00.063] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:03:00.654] [NewPayload] Handling new payload        height=20923721 hash=0x1486d150dbe52b6b3d74fe721742c4f0b7334d2eb7df80a425342ceb139c5933
[INFO] [10-08|22:03:00.654] [EngineBlockDownloader] Downloading PoS headers... hash=0x3a99219ae1debc5acaa7cf2c126d3ecd2192c778f08931fa68bc0db284d0b30b requestId=0
[INFO] [10-08|22:03:30.654] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:03:36.990] [NewPayload] Handling new payload        height=20923724 hash=0x9edf6ef37f34890e8fa5fda9709946cd3b0bcdb1bb5f58694eead671314f0e4e
[WARN] [10-08|22:03:37.412] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:03:49.403] [NewPayload] Handling new payload        height=20923725 hash=0xdf90087da58fc14931cb5d55f56073f0bcb1d38df2a500da502d3e7cb3361b94
[INFO] [10-08|22:03:49.403] [EngineBlockDownloader] Downloading PoS headers... hash=0x9edf6ef37f34890e8fa5fda9709946cd3b0bcdb1bb5f58694eead671314f0e4e requestId=0
[INFO] [10-08|22:04:19.248] [p2p] GoodPeers                          eth67=3 eth68=11
[INFO] [10-08|22:04:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=6.4GB sys=12.5GB
[INFO] [10-08|22:04:19.347] [mem] memory stats                       Rss=13.2GB Size=0B Pss=13.2GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.6GB Referenced=13.2GB Anonymous=9.6GB Swap=0B alloc=6.4GB sys=12.5GB
[INFO] [10-08|22:04:19.404] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:04:23.402] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:04:25.741] [NewPayload] Handling new payload        height=20923728 hash=0x5b24e5752572e2bf86ea01af923ad9204a10d3bddde39bca65636e3a5b85b568
[INFO] [10-08|22:04:25.741] [EngineBlockDownloader] Downloading PoS headers... hash=0xb972685074810d45e9b2b087d41fb46585ee00deb3af7ee2f462379257b7f575 requestId=0
[INFO] [10-08|22:04:55.742] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:04:58.810] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:05:01.153] [NewPayload] Handling new payload        height=20923731 hash=0x4898a6cb6b1dbafeda6903c37ed5ce78fefa4f38301aea25a4c68efa405701ff
[INFO] [10-08|22:05:01.153] [EngineBlockDownloader] Downloading PoS headers... hash=0x71913df0f952d8e0a5b567c1eb3688004c05bc3ccab9816315a31de4fb8140f4 requestId=0
[INFO] [10-08|22:05:31.154] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:05:40.593] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:05:49.294] [NewPayload] Handling new payload        height=20923735 hash=0x0099e3557a7321342349af141fd58bbe967c1b47e9f398ddd34e02515c92174d
[INFO] [10-08|22:05:49.294] [EngineBlockDownloader] Downloading PoS headers... hash=0x07ac3fc8003f296dafb481fd5398046db69e8199e77db71f73c1b4d585fb35fe requestId=0
[INFO] [10-08|22:06:19.295] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:06:31.762] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:06:38.475] [NewPayload] Handling new payload        height=20923739 hash=0xa7cb25698e772c99e99bc0ed75626cff3e3fd241c486f2d271ce0f675aedf921
[INFO] [10-08|22:06:38.475] [EngineBlockDownloader] Downloading PoS headers... hash=0x1c3d01a3c555ba0defbb3f7a8c216a3867d16205d0d82624a1576bf624736429 requestId=0
[INFO] [10-08|22:07:08.475] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:07:19.248] [p2p] GoodPeers                          eth68=11 eth67=4
[INFO] [10-08|22:07:19.347] [downloader] Downloaded PoS Headers      now=20883994 blk/sec=985.600
[INFO] [10-08|22:07:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.5GB sys=12.5GB
[INFO] [10-08|22:07:19.360] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.6GB Referenced=13.3GB Anonymous=9.6GB Swap=0B alloc=8.5GB sys=12.5GB
[WARN] [10-08|22:07:20.813] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:07:24.746] [NewPayload] Handling new payload        height=20923743 hash=0x2b49b65af7b9e7dd1d1e87050682422564717d26a94d87fc7be6d714f6d42b52
[INFO] [10-08|22:07:24.746] [EngineBlockDownloader] Downloading PoS headers... hash=0x068c5aca8a9bd240776aff9f83de3b65cac50f8e91246389e61d8f133ba01eb6 requestId=0
[INFO] [10-08|22:07:54.747] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:08:01.683] [NewPayload] Handling new payload        height=20923746 hash=0xee1699694bcd635abdbf8937e896281e1ab67ff239d6bf6f12298b0016cbb671
[WARN] [10-08|22:08:02.012] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:08:12.852] [NewPayload] Handling new payload        height=20923747 hash=0xbda7e05a875791f0b46eacec7e42a7b27afd90aa4bee981bca389e66ca36f91b
[INFO] [10-08|22:08:12.852] [EngineBlockDownloader] Downloading PoS headers... hash=0xee1699694bcd635abdbf8937e896281e1ab67ff239d6bf6f12298b0016cbb671 requestId=0
[WARN] [10-08|22:08:39.467] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:08:49.070] [NewPayload] Handling new payload        height=20923750 hash=0xad4c7b538ffe00f80de728b1ba1ef1644ddd6253d8de6cb31550eb8b5a2d20a2
[INFO] [10-08|22:08:49.070] [EngineBlockDownloader] Downloading PoS headers... hash=0x4ec7d0b8d74451d46ad681ddacfef510c67cf1e8935d6b8c330b02b9d3cfd9f0 requestId=0
[WARN] [10-08|22:09:07.894] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:09:12.632] [NewPayload] Handling new payload        height=20923752 hash=0xe297cf530ef2be0bd033f4743fc8419a9ca9171912523fa2c6e0a35bff1ecebc
[INFO] [10-08|22:09:12.632] [EngineBlockDownloader] Downloading PoS headers... hash=0xa1394ec5199d6c34e11481d42b9b87e70bcd34d87240210d1ceb08d1a0f0bf77 requestId=0
[WARN] [10-08|22:09:28.994] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:09:37.301] [NewPayload] Handling new payload        height=20923754 hash=0x093d1ba28265f7898f5ce7a837cf94d40da6f64e4bd851a17438892839486712
[INFO] [10-08|22:09:37.301] [EngineBlockDownloader] Downloading PoS headers... hash=0x6582c06c8d2f4e3605dc7879a92d75df47803d31ca2ef511cf1b81cb5cc64c98 requestId=0
[WARN] [10-08|22:09:53.943] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:10:01.372] [NewPayload] Handling new payload        height=20923756 hash=0x5b0677764d64daf85691cc27073c91a6ec9e7e613f197ca340809770775da982
[INFO] [10-08|22:10:01.372] [EngineBlockDownloader] Downloading PoS headers... hash=0x3e06cc75c9032c8986a590f42f9a1e1d9eee1e4aed1dccf29f8fb57f3b03c94e requestId=0
[INFO] [10-08|22:10:19.249] [p2p] GoodPeers                          eth68=14 eth67=4
[INFO] [10-08|22:10:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.2GB sys=12.5GB
[INFO] [10-08|22:10:19.359] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.6GB Referenced=13.3GB Anonymous=9.6GB Swap=0B alloc=5.2GB sys=12.5GB
[WARN] [10-08|22:10:19.861] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:10:24.708] [NewPayload] Handling new payload        height=20923758 hash=0x881f0483b5d2a49887032531855f90fdf21db080df34761568007ed656b63835
[INFO] [10-08|22:10:24.708] [EngineBlockDownloader] Downloading PoS headers... hash=0x3687630dc5852e5f6b9af4992e65ac40ef60df26753bc94c16dc35a8122e3c38 requestId=0
[WARN] [10-08|22:10:42.379] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:10:48.712] [NewPayload] Handling new payload        height=20923760 hash=0xe535799e1da953b791a55516be9de3261a401b4c63eb910c157861fdbb04ddb1
[INFO] [10-08|22:10:48.712] [EngineBlockDownloader] Downloading PoS headers... hash=0x6af6b07456d79f2dd4c357a9c1482ee9b6984a8ddf52f3c5039543c125604a7c requestId=0
[WARN] [10-08|22:11:07.037] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:11:12.917] [NewPayload] Handling new payload        height=20923762 hash=0x4d2f371b254b200de39574ef9592ae63eddaa14c2519d8ef873accfee13bd76b
[INFO] [10-08|22:11:12.917] [EngineBlockDownloader] Downloading PoS headers... hash=0x21ec7f3894288896d2834fea56ed8ab4acc033026f4b8be811d121e7b88ecde9 requestId=0
[WARN] [10-08|22:11:31.326] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:11:38.217] [NewPayload] Handling new payload        height=20923764 hash=0xb690836b332f3f5496c81ff5f727a99295fd887c24167d4d332631aab351878e
[INFO] [10-08|22:11:38.217] [EngineBlockDownloader] Downloading PoS headers... hash=0xe5c30b657db4b78d144ed84e53266528de11b6b5f3a54627ad6c82cea3874921 requestId=0
[WARN] [10-08|22:11:54.529] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:12:01.787] [NewPayload] Handling new payload        height=20923766 hash=0x1ed9a78fde48e79b545f11e539ba24c9d0679042b55baa722f46ed3ccc54ecdb
[INFO] [10-08|22:12:01.787] [EngineBlockDownloader] Downloading PoS headers... hash=0x94fb357177e7954ece6ea91103a5aa3632d90f9f79b5d72bc0d58f79b4691a41 requestId=0
[WARN] [10-08|22:12:19.178] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:12:24.713] [NewPayload] Handling new payload        height=20923768 hash=0x0af7fe67aae1d8b0bda7736fc9f696819eca24bafcb61540bd2cec89b0146fb2
[INFO] [10-08|22:12:24.714] [EngineBlockDownloader] Downloading PoS headers... hash=0x55389313679242bad6d18a9f3d0f9da41b759df630149d1dc53f7192d8d09160 requestId=0
[WARN] [10-08|22:12:43.181] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:12:48.414] [NewPayload] Handling new payload        height=20923770 hash=0x603f806c708d2574ab347721d1262a19f1472c37528e6b489f4adfa0d327dee0
[INFO] [10-08|22:12:48.414] [EngineBlockDownloader] Downloading PoS headers... hash=0xe1555baf5a7801612f7eb820bd3f17d4ac7ab253be1468d6d5e0882bf83e714e requestId=0
[WARN] [10-08|22:13:06.959] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:13:13.124] [NewPayload] Handling new payload        height=20923772 hash=0x40d85352ddaeb1ab829136051eaee9b3942a8add6c7c4d801dea2278f2330834
[INFO] [10-08|22:13:13.124] [EngineBlockDownloader] Downloading PoS headers... hash=0x9a9bbb7ef0f080b46f2d6a133129e6ec53d576c5c4d73339d1db3f362614c1a9 requestId=0
[INFO] [10-08|22:13:19.248] [p2p] GoodPeers                          eth68=14 eth67=4
[INFO] [10-08|22:13:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.0GB sys=12.5GB
[INFO] [10-08|22:13:19.354] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.7GB PrivateDirty=9.6GB Referenced=13.3GB Anonymous=9.6GB Swap=0B alloc=8.0GB sys=12.5GB
[WARN] [10-08|22:13:29.244] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:13:38.325] [NewPayload] Handling new payload        height=20923774 hash=0xb9e9b1d41257266080d3558fff3f0a01a15ece30e1baa1e81eacfcf8bb4533af
[INFO] [10-08|22:13:38.325] [EngineBlockDownloader] Downloading PoS headers... hash=0x2b5d248ef15ffeb68c1f0c040e6f7102bbaa1b254a4a93aad0e126e02849028d requestId=0
[WARN] [10-08|22:13:56.928] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:14:00.761] [NewPayload] Handling new payload        height=20923776 hash=0x09559894a4336fc659b0beae1caeba8fd559d4a31378f0f0884478c44fb483d4
[INFO] [10-08|22:14:00.761] [EngineBlockDownloader] Downloading PoS headers... hash=0x5d005681e042f7ca5006afe779410a9e00913f6f040983de76b42bf2c2f85963 requestId=0
[WARN] [10-08|22:14:16.653] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:14:24.972] [NewPayload] Handling new payload        height=20923778 hash=0xeb4caa1ff93c2c2e2c494a34e4b56e0a29365c6f001c90545908378e5055f974
[INFO] [10-08|22:14:24.972] [EngineBlockDownloader] Downloading PoS headers... hash=0x3823b4931846a300d95b839b21ea3cffa57129d7c01534c52ea170b4473a256f requestId=0
[WARN] [10-08|22:14:46.192] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:14:46.536] [EngineBlockDownloader] Downloading PoS headers... hash=0x33b826501b461187371bbee6035c9e1ccab283ff34090b5ac769b9655a0b7f74 requestId=0
[WARN] [10-08|22:15:06.897] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:15:14.359] [NewPayload] Handling new payload        height=20923782 hash=0xeaeec228f00f619643458c36d6c57507431d6d8e14f52db69be3a531b996384e
[INFO] [10-08|22:15:14.359] [EngineBlockDownloader] Downloading PoS headers... hash=0xfe4ed763b750e12cecd76c7d3ecf19f5daaa1c4e390f0cbff8e5edf0880334b7 requestId=0
[WARN] [10-08|22:15:40.181] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:15:48.793] [NewPayload] Handling new payload        height=20923785 hash=0x805f8b1a98cd5da47a5ec64f48418fa39b42ef82453babc80c33de3e28770efe
[INFO] [10-08|22:15:48.793] [EngineBlockDownloader] Downloading PoS headers... hash=0x9e34eb6d4ccd36cbfc4412eed40e9c3a4442e4e9341276b93c82deae4ad7efea requestId=0
[WARN] [10-08|22:16:16.586] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:16:19.249] [p2p] GoodPeers                          eth68=14 eth67=6
[INFO] [10-08|22:16:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.2GB sys=12.5GB
[INFO] [10-08|22:16:19.349] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.6GB Referenced=10.0GB Anonymous=9.6GB Swap=0B alloc=5.2GB sys=12.5GB
[INFO] [10-08|22:16:24.898] [NewPayload] Handling new payload        height=20923788 hash=0x818f1f4ebed7e05f12bb48870fae3876fbd368de3a432ca27f42141e0b6dbdf8
[INFO] [10-08|22:16:24.898] [EngineBlockDownloader] Downloading PoS headers... hash=0x4557e9a9402537960bbf8972171c99671bd264b0c729641761b2b8012336cfa6 requestId=0
[WARN] [10-08|22:16:50.762] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:17:01.213] [NewPayload] Handling new payload        height=20923791 hash=0x94a9d8eeda173ae0214383627aa8045a33b48eb7c5de6ddea1356962245463b7
[INFO] [10-08|22:17:01.213] [EngineBlockDownloader] Downloading PoS headers... hash=0xb63a33671107d35b77a60fb1927effc12ea89b4f35fa40bc3b86c99a0ba88a0b requestId=0
[INFO] [10-08|22:17:25.707] [NewPayload] Handling new payload        height=20923793 hash=0x8f402d59d729b2ae7be517d34938de49fb75664572596f23be196c710932500b
[WARN] [10-08|22:17:25.741] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:17:26.001] [EngineBlockDownloader] Downloading PoS headers... hash=0x8f402d59d729b2ae7be517d34938de49fb75664572596f23be196c710932500b requestId=0
[WARN] [10-08|22:17:53.661] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:18:01.014] [NewPayload] Handling new payload        height=20923796 hash=0x91f3dfa036622bbd94205418881a4bc385aad5b47049983bdf3a63aea7325b00
[INFO] [10-08|22:18:01.014] [EngineBlockDownloader] Downloading PoS headers... hash=0x775afb8c9f81684b39616fcd7011b35a02b5efea66fc1d66bd4e82334d35e52d requestId=0
[WARN] [10-08|22:18:25.360] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:18:38.140] [NewPayload] Handling new payload        height=20923799 hash=0x62c5965d342ec2f7a73810e374d342f04e14822bf41661d606f339bf9b184541
[INFO] [10-08|22:18:38.140] [EngineBlockDownloader] Downloading PoS headers... hash=0xdd4fd5981a6b15b9350afae59da2000d79f43202454049a0eec1c596c5bf7983 requestId=0
[WARN] [10-08|22:19:02.712] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:19:13.149] [NewPayload] Handling new payload        height=20923802 hash=0x7fd9f92b0c605cec5fc6572eb9257858ad06add7e606b959a967253525f0375b
[INFO] [10-08|22:19:13.149] [EngineBlockDownloader] Downloading PoS headers... hash=0xbb4a851eb60d42c1a37a615cb6ab54692f6eef39d093056a12b0dbba8f7da9d3 requestId=0
[INFO] [10-08|22:19:19.249] [p2p] GoodPeers                          eth68=16 eth67=7
[INFO] [10-08|22:19:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=9.1GB sys=12.5GB
[INFO] [10-08|22:19:19.363] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.6GB Referenced=10.1GB Anonymous=9.6GB Swap=0B alloc=9.1GB sys=12.5GB
[WARN] [10-08|22:19:39.887] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:19:49.506] [NewPayload] Handling new payload        height=20923805 hash=0x844f1233a852bb78e8594b6059dae4b2731a47bd9cb33d836481f7c686d9ada3
[INFO] [10-08|22:19:49.506] [EngineBlockDownloader] Downloading PoS headers... hash=0xafc034def785ffa6fd0e5f8128f47e0a37eeedab1833a1ff90c3e0aa67ad5b97 requestId=0
[WARN] [10-08|22:20:14.985] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:20:24.813] [NewPayload] Handling new payload        height=20923808 hash=0xe14b8a41443d068fb86cd31a4878726e3072534df183b2cdc85633b72a7b0232
[INFO] [10-08|22:20:24.813] [EngineBlockDownloader] Downloading PoS headers... hash=0x8f2480570528dda443689dbf89621d066c677f45824b4822c9d26e793a976ff8 requestId=0
[INFO] [10-08|22:20:54.813] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:20:58.153] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:21:00.939] [NewPayload] Handling new payload        height=20923811 hash=0xaac11c379071acd1f1be2e3ea590877a1a9ffb32f1794069a82a4c7d778321fc
[INFO] [10-08|22:21:00.940] [EngineBlockDownloader] Downloading PoS headers... hash=0x63210cb8330dbf3adec83e469e905ccf625948c7fb270b912061a0abf798c71c requestId=0
[WARN] [10-08|22:21:29.228] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:21:36.816] [NewPayload] Handling new payload        height=20923814 hash=0x553cd3990ad1f90f17c4b582f78753bec180638e6970e52fc96faf5ebe387d60
[INFO] [10-08|22:21:36.816] [EngineBlockDownloader] Downloading PoS headers... hash=0xd6f077af33cc2c8307d548417c2e0210c3bae5cfe98d4741c223bc0ef66ff819 requestId=0
[INFO] [10-08|22:22:06.816] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:22:08.828] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:22:13.840] [NewPayload] Handling new payload        height=20923817 hash=0xdb1bb77a5d1da48c733c16b7b25989973d4d209ac72c77fd57160752fd0816fe
[INFO] [10-08|22:22:13.841] [EngineBlockDownloader] Downloading PoS headers... hash=0x51c81c55d28ab00a1219e9f6033083d9887484336108b5af81487ba4a0045648 requestId=0
[INFO] [10-08|22:22:19.249] [p2p] GoodPeers                          eth68=15 eth67=7
[INFO] [10-08|22:22:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.2GB sys=12.5GB
[INFO] [10-08|22:22:19.376] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=8.2GB sys=12.5GB
[INFO] [10-08|22:22:43.841] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:22:44.596] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:22:49.143] [NewPayload] Handling new payload        height=20923820 hash=0x236e8ea93d80c9174b9dab1d2639a7389db760407742b5a25bae22ba27be1217
[INFO] [10-08|22:22:49.143] [EngineBlockDownloader] Downloading PoS headers... hash=0x82ad4b72f879f7ac6bafb3474f26c97bf0b1f1451d2f1f2c9f7297382c70c303 requestId=0
[INFO] [10-08|22:23:19.144] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:23:25.838] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:23:37.431] [NewPayload] Handling new payload        height=20923824 hash=0x9fe34b2f54cd1ccbc5021362ea130d9976b7b5656f083f5982e49d69fbdb6488
[INFO] [10-08|22:23:37.431] [EngineBlockDownloader] Downloading PoS headers... hash=0xe4b3b7624953d77e52889e5d5d408188fc63a4fbeef1533a330776d55323b0fc requestId=0
[WARN] [10-08|22:23:57.660] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:24:00.845] [NewPayload] Handling new payload        height=20923826 hash=0x3b9f70e68ca459b699ed6951c68a2624839a8671be40e34af43af41174edddb9
[INFO] [10-08|22:24:00.845] [EngineBlockDownloader] Downloading PoS headers... hash=0x1f260b9f7472361d9f959ae2af0cd254fb105033644cd7abd9ccb36e917b9c75 requestId=0
[WARN] [10-08|22:24:23.453] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:24:24.789] [NewPayload] Handling new payload        height=20923828 hash=0x862b9e73232a4563888a059a73828e764cfb5c59d7087d9fbb9238652c2f7f74
[INFO] [10-08|22:24:24.789] [EngineBlockDownloader] Downloading PoS headers... hash=0x72430e27366213e3cc02a55739315bf22ae7da63128d04ed9bd5ba1f80758a46 requestId=0
[WARN] [10-08|22:24:44.390] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:24:48.739] [NewPayload] Handling new payload        height=20923830 hash=0xb44f859469cf924b441ecff23ed11f70d10e93a69ac7b24c6475907cd3f23f30
[INFO] [10-08|22:24:48.740] [EngineBlockDownloader] Downloading PoS headers... hash=0xad9062561f255637d6ad77996090cbe0e22ac97c646892ab89a980f833cbf9dd requestId=0
[WARN] [10-08|22:25:08.843] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:25:13.143] [NewPayload] Handling new payload        height=20923832 hash=0x7c7e48e08d8cad38564cb6477b1dfbc00c31709a7bef8cae39c984bfb638b6f9
[INFO] [10-08|22:25:13.143] [EngineBlockDownloader] Downloading PoS headers... hash=0x94f6a2b27d0ed9034b158b77d3e5732ad1db8ff91d811aad6132d598bf28c9af requestId=0
[INFO] [10-08|22:25:19.323] [p2p] GoodPeers                          eth68=18 eth67=8
[INFO] [10-08|22:25:19.424] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=9.1GB sys=12.5GB
[INFO] [10-08|22:25:20.238] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=6.3GB sys=12.5GB
[WARN] [10-08|22:25:34.563] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:25:37.046] [NewPayload] Handling new payload        height=20923834 hash=0x0407b0b365d9831d2ccc38e96fa61f4cf1e5d89c468b10e03ccddf6eb6fb1f9b
[INFO] [10-08|22:25:37.046] [EngineBlockDownloader] Downloading PoS headers... hash=0xce5a614cc99cde6e6654fd2c458064929dbbe24de8ce55973c4cce86ff1dc7b5 requestId=0
[WARN] [10-08|22:25:55.934] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:26:01.093] [NewPayload] Handling new payload        height=20923836 hash=0xae17da76ca4454b033ffde8a54484117f7bb8336c8fa364a8fcd744336f0448a
[INFO] [10-08|22:26:01.093] [EngineBlockDownloader] Downloading PoS headers... hash=0x581e58a7ec46c97d71c655ba9eab8fd733d6e10e88ca5cffb804c009c7cc3089 requestId=0
[WARN] [10-08|22:26:19.485] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:26:24.824] [NewPayload] Handling new payload        height=20923838 hash=0x8231f5a16b8aa6126ce7fa937f4d731d8a69c10a6d85faa2a4523c86eea40e9c
[INFO] [10-08|22:26:24.825] [EngineBlockDownloader] Downloading PoS headers... hash=0xa9d2b3ef6d6f377d945ba7fd16749d60fd5fc8e904abf07d9c7ab808983ff695 requestId=0
[WARN] [10-08|22:26:45.865] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:26:48.811] [NewPayload] Handling new payload        height=20923840 hash=0x7e9f87eff4c23d0121f73a276968359b34c6575f7930e630d02a788b5843fa19
[INFO] [10-08|22:26:48.811] [EngineBlockDownloader] Downloading PoS headers... hash=0x127bd4e095ee7779e417137174fd423c4d492c863d7f27e184bc4e92b102634c requestId=0
[WARN] [10-08|22:27:07.633] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:27:12.677] [NewPayload] Handling new payload        height=20923842 hash=0xefcfd9049cdd0b38377d30485b0f2975ce7ceea07e281619d0e02dd411a0adaa
[INFO] [10-08|22:27:12.677] [EngineBlockDownloader] Downloading PoS headers... hash=0x363e137a11933999ff75314c91dc5b9d26d330570f1dc31da2a5340415be5285 requestId=0
[WARN] [10-08|22:27:33.539] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:27:34.533] [EngineBlockDownloader] Downloading PoS headers... hash=0xd2a6024d0f369735f2422e8f27056f2db56d889ef38f58f5bd03c872de96bc1b requestId=0
[WARN] [10-08|22:27:54.545] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:28:01.075] [NewPayload] Handling new payload        height=20923846 hash=0xa6102f60d8bcee91acb06c610ed22800f8ff8836b52458f81f103720921c3947
[INFO] [10-08|22:28:01.075] [EngineBlockDownloader] Downloading PoS headers... hash=0x98e0a9f47978fb97210f2f51ccde78874c0e53b9604dedb9daac08d226c327e1 requestId=0
[INFO] [10-08|22:28:19.248] [p2p] GoodPeers                          eth68=18 eth67=8
[INFO] [10-08|22:28:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.8GB sys=12.5GB
[INFO] [10-08|22:28:19.356] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.8GB sys=12.5GB
[WARN] [10-08|22:28:22.598] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:28:26.362] [NewPayload] Handling new payload        height=20923848 hash=0x83f0b0779dbbc5673ff2bcfbb8a57151a6d78a4a4ad141e43b0811bd5e2e2407
[INFO] [10-08|22:28:26.362] [EngineBlockDownloader] Downloading PoS headers... hash=0x4475ec4ca472140b5d1faf765881ffba773f2793cc0f03b8de2dd3d4077dfd17 requestId=0
[WARN] [10-08|22:28:47.887] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:28:49.070] [NewPayload] Handling new payload        height=20923850 hash=0x11c3f5fd29a4c64a486e654db6911ba3af0472fb76db9e720a0e58884f63db03
[INFO] [10-08|22:28:49.070] [EngineBlockDownloader] Downloading PoS headers... hash=0x6f3305264fb6dacd7b6952099f967d69d80b028d53a0e0ace38a273bdb890a09 requestId=0
[WARN] [10-08|22:29:10.399] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:29:14.589] [NewPayload] Handling new payload        height=20923852 hash=0xddcd47325ab7f7606e7f5c69b4300d4500d8aff8539b680451a1e4218633a645
[INFO] [10-08|22:29:14.589] [EngineBlockDownloader] Downloading PoS headers... hash=0xed26d97d22e7cc3b4d62f6a90c1670c7ce2f64a331417010c12b4280a4000fa4 requestId=0
[WARN] [10-08|22:29:36.951] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:29:38.670] [NewPayload] Handling new payload        height=20923854 hash=0x9168da5e1bd192803d2bb775b368ab2ca8d71a5754827ee79f364bfdc1838630
[INFO] [10-08|22:29:38.670] [EngineBlockDownloader] Downloading PoS headers... hash=0x1b4dee9eaec4508972bc4747d997dd939300de73fedddf91526d4b219873a7ab requestId=0
[WARN] [10-08|22:30:01.038] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:30:13.240] [NewPayload] Handling new payload        height=20923857 hash=0x2cd209dffbea0b64d7679794577de69f3627edfd6d0b82b4312e0602cb25daa6
[INFO] [10-08|22:30:13.240] [EngineBlockDownloader] Downloading PoS headers... hash=0x69c2244693e231104ff8560ca2e3bf77572a12856a28ac748dbd50224c0ed1b0 requestId=0
[WARN] [10-08|22:30:35.001] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:30:36.826] [NewPayload] Handling new payload        height=20923859 hash=0x6637b231dda988501131bf710c10442659e652ea21fff90f2887e47301b628d6
[INFO] [10-08|22:30:36.826] [EngineBlockDownloader] Downloading PoS headers... hash=0xa3eb9b82935c97f6ae932432eca43817d726e78796b9118d0833f30360fa3f87 requestId=0
[WARN] [10-08|22:30:58.466] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:31:00.937] [NewPayload] Handling new payload        height=20923861 hash=0xc54133fd9389cf567375b38de9179f2eac60596bb511b8d24880afb99bcceff9
[INFO] [10-08|22:31:00.937] [EngineBlockDownloader] Downloading PoS headers... hash=0x240856a195c0d5c383556633ca019f01325181efa243aef49e50fc976b193178 requestId=0
[INFO] [10-08|22:31:19.248] [p2p] GoodPeers                          eth68=18 eth67=9
[INFO] [10-08|22:31:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.4GB sys=12.5GB
[INFO] [10-08|22:31:19.372] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=5.4GB sys=12.5GB
[WARN] [10-08|22:31:23.346] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:31:24.668] [NewPayload] Handling new payload        height=20923863 hash=0xa3ed7df63c4553aa79362712eb9c250109ba82f34ea78a9725fa2b72c50b5fd1
[INFO] [10-08|22:31:24.668] [EngineBlockDownloader] Downloading PoS headers... hash=0xeae3db6d08d7d41a0782d26f8b68e030d20ebfdbdf87a2ea8513e8e979bb8928 requestId=0
[WARN] [10-08|22:31:45.172] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:31:49.713] [NewPayload] Handling new payload        height=20923865 hash=0xd12d6b9833e0b79c031aa9089fb06e6a19af2932c7e06e7f183d321e2c11d2b6
[INFO] [10-08|22:31:49.713] [EngineBlockDownloader] Downloading PoS headers... hash=0xf874ceb226ccd694015360d1ea9c904a7e59690d7a37366fd569f946621aaf71 requestId=0
[WARN] [10-08|22:32:11.842] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:32:12.340] [NewPayload] Handling new payload        height=20923867 hash=0x4110d44c697dbc6cd9bf7432a743f297b698e0f8144a176c9560ebdde24158e2
[INFO] [10-08|22:32:12.340] [EngineBlockDownloader] Downloading PoS headers... hash=0x96173fb9ee6c4c6e8b26d2b04b9bec3a89fab6a9d62c9222ae601540904ae39b requestId=0
[WARN] [10-08|22:32:36.104] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:32:36.679] [NewPayload] Handling new payload        height=20923869 hash=0x5a991f6d8c8cb764cb2a6b384db9ae922b16d2b2a33d8ad24ceee52570d59208
[INFO] [10-08|22:32:36.679] [EngineBlockDownloader] Downloading PoS headers... hash=0x58030a9a4119ab99a5c80cb671a4ee7511353c11c492723c51b5d7f76d18968e requestId=0
[WARN] [10-08|22:33:00.270] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:33:00.701] [NewPayload] Handling new payload        height=20923871 hash=0x684aa687dbce3790827e5c42f486061cfa9ea57d329d8b16b1e06f4c2b286187
[INFO] [10-08|22:33:00.701] [EngineBlockDownloader] Downloading PoS headers... hash=0xbb6045b45d9c910c1f9a3fe028ae142aa8b5ce15f3eda9c5a09cd50c0aa4c443 requestId=0
[INFO] [10-08|22:33:26.300] [NewPayload] Handling new payload        height=20923873 hash=0x5777df296d582ce2c8d382df45c4665e38f854396bf54b02ed67e4490ea5e75c
[WARN] [10-08|22:33:26.571] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:33:36.759] [NewPayload] Handling new payload        height=20923874 hash=0x6530d4c42a28c3bff1c504a03f7ef6ba3487502dd4d07115e0ee8348171908ce
[INFO] [10-08|22:33:36.759] [EngineBlockDownloader] Downloading PoS headers... hash=0x5777df296d582ce2c8d382df45c4665e38f854396bf54b02ed67e4490ea5e75c requestId=0
[WARN] [10-08|22:34:01.187] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:34:02.793] [EngineBlockDownloader] Downloading PoS headers... hash=0xee22d166a65eb35ca2d379a7d0635229c145e3a0fdf0782f89385655ff8477fb requestId=0
[INFO] [10-08|22:34:19.248] [p2p] GoodPeers                          eth68=18 eth67=11
[INFO] [10-08|22:34:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.5GB sys=12.5GB
[INFO] [10-08|22:34:19.371] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.5GB sys=12.5GB
[WARN] [10-08|22:34:28.328] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:34:36.847] [NewPayload] Handling new payload        height=20923879 hash=0x2bebb53124f9bb4b95860c5ef6a6f96e5b68cdc347cf84361441b060b94a8efa
[INFO] [10-08|22:34:36.847] [EngineBlockDownloader] Downloading PoS headers... hash=0x59d881f62174b5d0e223492a555177c58d1b1961ec7a1bbcffec0450280ed673 requestId=0
[WARN] [10-08|22:35:04.001] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:35:12.783] [NewPayload] Handling new payload        height=20923882 hash=0x069967da1e2310270c04cac7d4e44cb323346b1dd5ab00a8a3a41dccd26d2349
[INFO] [10-08|22:35:12.783] [EngineBlockDownloader] Downloading PoS headers... hash=0xcc2d89a5d2a29e5e4a22f341b574e77f90dfcf9eb54fb2920398d5af1f27eeca requestId=0
[INFO] [10-08|22:35:36.691] [NewPayload] Handling new payload        height=20923884 hash=0x77a021843773d34d27b005cbc82ac052269f9617c4d9fd79617087f58da1e6dc
[WARN] [10-08|22:35:37.174] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:35:48.796] [NewPayload] Handling new payload        height=20923885 hash=0x7118ecd0aef0498be6315f0946ce55e08ec01ee44a3008b92cddfbae4cf2e2dd
[INFO] [10-08|22:35:48.796] [EngineBlockDownloader] Downloading PoS headers... hash=0x77a021843773d34d27b005cbc82ac052269f9617c4d9fd79617087f58da1e6dc requestId=0
[WARN] [10-08|22:36:13.371] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:36:24.603] [NewPayload] Handling new payload        height=20923888 hash=0x5c87bfd87161b59b1ffff191715856296eb07a135a4f053a63ce04906c1ba592
[INFO] [10-08|22:36:24.603] [EngineBlockDownloader] Downloading PoS headers... hash=0xfa54b605cc9e25ada1d6f9e56e6b5c1002fc56135950524ec4e16262e32879ec requestId=0
[WARN] [10-08|22:36:51.990] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:37:00.715] [NewPayload] Handling new payload        height=20923891 hash=0xd070cf282edc830189409a42b7d1778c5ee01f8b18e3cacc154d780b9de93730
[INFO] [10-08|22:37:00.715] [EngineBlockDownloader] Downloading PoS headers... hash=0x56bd2eb4462e074d1ff5bacfdcccfe91a68be78ea072aad8e12e5999fcd784ca requestId=0
[INFO] [10-08|22:37:19.248] [p2p] GoodPeers                          eth68=18 eth67=10
[INFO] [10-08|22:37:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=6.9GB sys=12.5GB
[INFO] [10-08|22:37:19.369] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=6.9GB sys=12.5GB
[WARN] [10-08|22:37:24.527] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:37:24.787] [NewPayload] Handling new payload        height=20923893 hash=0x3b6ba0879ee15c94011516b9ef3e2930e8876ff2442f6a41dd4aeb5f5072fb8e
[INFO] [10-08|22:37:24.788] [EngineBlockDownloader] Downloading PoS headers... hash=0xb0d3ac9855267658d09560a2fad65bc53d74cf235e067582fcf0a24f245e2df8 requestId=0
[INFO] [10-08|22:37:48.794] [NewPayload] Handling new payload        height=20923895 hash=0x604ebb020106051798d07135b29301dc10b75241fb2938020eaa23d8665d9ef9
[WARN] [10-08|22:37:48.925] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:38:00.817] [NewPayload] Handling new payload        height=20923896 hash=0xc027da7d77b055b15c4ec8a679d5dd600557b5faa657504aaf34a710e4ba5cc7
[INFO] [10-08|22:38:00.818] [EngineBlockDownloader] Downloading PoS headers... hash=0x604ebb020106051798d07135b29301dc10b75241fb2938020eaa23d8665d9ef9 requestId=0
[WARN] [10-08|22:38:28.549] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:38:36.603] [NewPayload] Handling new payload        height=20923899 hash=0x17e7be9f1439dd1b5f6e4cf02afb042f2d7c0126ce417271c53c9d9fdea234e2
[INFO] [10-08|22:38:36.603] [EngineBlockDownloader] Downloading PoS headers... hash=0x80eb8cc91c89ad88a8049e7288f006a9ff9cff712237a3abc6e3421450a89a58 requestId=0
[WARN] [10-08|22:39:01.046] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:39:01.287] [NewPayload] Handling new payload        height=20923901 hash=0x170aea743200ab08c64abf1ae2163dba5dc3d29acce9df9d234ab308aba46124
[INFO] [10-08|22:39:01.287] [EngineBlockDownloader] Downloading PoS headers... hash=0x886360922d7bd9486f5e0ab48112aa1280298694dbe4ce67ad135080c04fdc34 requestId=0
[WARN] [10-08|22:39:27.524] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:39:36.821] [NewPayload] Handling new payload        height=20923904 hash=0x08b926132a09fe45499c0da4b8824e41d3b26f449d52fe4e83746532362456ab
[INFO] [10-08|22:39:36.821] [EngineBlockDownloader] Downloading PoS headers... hash=0xc780516d16cd18e794f8d9ad76b5d5be784efd13de261acf84e8923a0368b028 requestId=0
[INFO] [10-08|22:40:00.949] [NewPayload] Handling new payload        height=20923906 hash=0xbbae22989fbb5a89064c5395eb57b9b86bc97d0082c13096cc71a7f7f7ad4cb4
[WARN] [10-08|22:40:01.040] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:40:01.043] [EngineBlockDownloader] Downloading PoS headers... hash=0xbbae22989fbb5a89064c5395eb57b9b86bc97d0082c13096cc71a7f7f7ad4cb4 requestId=0
[INFO] [10-08|22:40:19.248] [p2p] GoodPeers                          eth68=19 eth67=11
[INFO] [10-08|22:40:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.6GB sys=12.5GB
[INFO] [10-08|22:40:19.351] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.6GB sys=12.5GB
[INFO] [10-08|22:40:31.043] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:40:33.744] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:40:36.676] [NewPayload] Handling new payload        height=20923909 hash=0x9cf5571210abbbd9d9a4c34d9eaa2f7b343d7198050529abee737921f5e951ca
[INFO] [10-08|22:40:36.676] [EngineBlockDownloader] Downloading PoS headers... hash=0x3d277eac14823602ec9809cc51aa8e4feaf653693227967941f674b4425c90af requestId=0
[INFO] [10-08|22:41:06.676] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:41:17.123] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:41:24.794] [NewPayload] Handling new payload        height=20923913 hash=0x1dbdfb5695d1db3cc367d65bc3446963f811030c3579c6ee0cc9d3d093043bc3
[INFO] [10-08|22:41:24.794] [EngineBlockDownloader] Downloading PoS headers... hash=0xc3cabbb5238a3dde0410d6677b06c79f518f19399abed0a68f7059573ec47327 requestId=0
[INFO] [10-08|22:41:54.795] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:42:00.769] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:42:00.982] [NewPayload] Handling new payload        height=20923916 hash=0x4b2908337771e720f78c467f968b6602eb6a3d8cd5237a0b9e3330f34499b125
[INFO] [10-08|22:42:00.982] [EngineBlockDownloader] Downloading PoS headers... hash=0xcb4f58f832cea5463a101b3801ea6297e6a6abb8f658cc8b3a253fffe3ad1868 requestId=0
[INFO] [10-08|22:42:30.982] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:42:40.881] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:42:49.271] [NewPayload] Handling new payload        height=20923920 hash=0x65705c19fdbb48298b2004e4a2d7ed0e1701edbbadd35c573240a45ecb1269fc
[INFO] [10-08|22:42:49.271] [EngineBlockDownloader] Downloading PoS headers... hash=0x7f5ac7b82d832a947d6266bc05859189de575999b960091c0ac6d8b56b56dfa0 requestId=0
[INFO] [10-08|22:43:19.249] [p2p] GoodPeers                          eth68=22 eth67=9
[INFO] [10-08|22:43:19.272] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:43:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=5.4GB sys=12.5GB
[INFO] [10-08|22:43:19.366] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=5.4GB sys=12.5GB
[WARN] [10-08|22:43:42.067] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:43:50.559] [NewPayload] Handling new payload        height=20923925 hash=0xcaadb1c579cfa8a8040a63db7354706474e3f1f2936d8a4d554cdc87cc5b12db
[INFO] [10-08|22:43:50.559] [EngineBlockDownloader] Downloading PoS headers... hash=0x2d433aa1c9b567533104a8dd000501c6e01ae1a45c6ac9d059da67449ed74dd1 requestId=0
[INFO] [10-08|22:44:20.559] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:44:35.836] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:44:37.232] [NewPayload] Handling new payload        height=20923929 hash=0xad74366168222a3a079eedce2bfbb128ea997b2a5956e2e082fdfa00f1ab47e1
[INFO] [10-08|22:44:37.232] [EngineBlockDownloader] Downloading PoS headers... hash=0x3a71fe6aeafcbb43d35cc058d9a582f3c05aefac7edd550e468b37f38d1b861d requestId=0
[INFO] [10-08|22:45:07.233] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:45:18.780] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:45:25.209] [NewPayload] Handling new payload        height=20923933 hash=0x28b1d8bc4fa93cf32870c327d630612baaff3beba78d51c37a8a5c8a5610be5e
[INFO] [10-08|22:45:25.209] [EngineBlockDownloader] Downloading PoS headers... hash=0xb82d1fb054f8623700b1c63b15f16bdcfff3dbf4fc26a932516315a0007a2f3c requestId=0
[INFO] [10-08|22:45:55.209] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:46:06.302] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:46:12.592] [NewPayload] Handling new payload        height=20923937 hash=0x75cac04a9c84bdaa7da8fb0e9fd1ba3d77bc005a8853d189905f81715f9a073f
[INFO] [10-08|22:46:12.592] [EngineBlockDownloader] Downloading PoS headers... hash=0x9b12229690f3c3501345ecb4e3f94edecabd8ebd1d7f0caf49a25695f7b1f878 requestId=0
[INFO] [10-08|22:46:19.248] [p2p] GoodPeers                          eth68=20 eth67=9
[INFO] [10-08|22:46:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.6GB sys=12.5GB
[INFO] [10-08|22:46:19.351] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.6GB sys=12.5GB
[INFO] [10-08|22:46:42.593] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:46:48.823] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:46:48.895] [NewPayload] Handling new payload        height=20923940 hash=0xa16a9431672f545a59ccb9bbcc47b6925f9fd0ef047011926ea8598b57485139
[INFO] [10-08|22:46:48.895] [EngineBlockDownloader] Downloading PoS headers... hash=0x347935a6b988a407e259bf67299564a7b3f7898e574eeec20084d09754c2461b requestId=0
[WARN] [10-08|22:47:12.144] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:47:12.813] [NewPayload] Handling new payload        height=20923942 hash=0x2cc106ea1344b7ca352d42331d84df23b07ca0eae7a80670551ac6ca47439422
[INFO] [10-08|22:47:12.813] [EngineBlockDownloader] Downloading PoS headers... hash=0xb7a330928f71a5bd056fc4d229ce5734e570fb775a1fdb5034c8c84eb6cd4e63 requestId=0
[WARN] [10-08|22:47:32.726] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:47:36.923] [NewPayload] Handling new payload        height=20923944 hash=0xc42216eeafaf5fd008216d01fae0f9718e39ac8396ff1966bc7efa18db9568ac
[INFO] [10-08|22:47:36.923] [EngineBlockDownloader] Downloading PoS headers... hash=0x35dadaec3f9e96fb1015c989d5878a2ee67f0951c26b6974499dee6ab31e5f86 requestId=0
[WARN] [10-08|22:47:58.482] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:48:01.353] [NewPayload] Handling new payload        height=20923946 hash=0xf00979c32c3aec8b990cdf1f462d1da4cf680e452aa912e2ad7f41bdb029c115
[INFO] [10-08|22:48:01.353] [EngineBlockDownloader] Downloading PoS headers... hash=0xce3fc623a40cd48e09ee4c8cc0ca45358489f970e64521ea73bfa3e7fcc71955 requestId=0
[INFO] [10-08|22:48:31.353] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:48:36.265] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:48:36.860] [NewPayload] Handling new payload        height=20923949 hash=0x170999e506f6033944ea82df403fa390fe8e3ce6d3fdb90580fd3303ba1e7b25
[INFO] [10-08|22:48:36.860] [EngineBlockDownloader] Downloading PoS headers... hash=0xb2dd49144fe899e13dc0325eac496e2c3671a2aae1ea99f000e1a768980304f2 requestId=0
[INFO] [10-08|22:49:06.861] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:49:10.801] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:49:12.742] [NewPayload] Handling new payload        height=20923952 hash=0x21c9309adc496d63b2b9852bd7df4e47a4bb4bd238b7446d8aa626c86b77f37b
[INFO] [10-08|22:49:12.742] [EngineBlockDownloader] Downloading PoS headers... hash=0x17ed36266d819f7206b6479c1e7a96e87cbec64fe4616b783d18a86d0864c484 requestId=0
[INFO] [10-08|22:49:19.248] [p2p] GoodPeers                          eth67=9 eth68=21
[INFO] [10-08|22:49:19.346] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=8.5GB sys=12.5GB
[INFO] [10-08|22:49:19.355] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=8.5GB sys=12.5GB
[INFO] [10-08|22:49:42.742] [EngineBlockDownloader] Waiting for headers download to finish
[INFO] [10-08|22:49:48.726] [NewPayload] Handling new payload        height=20923955 hash=0x357e6ff715b7e21faaa4d4ea18593270afd00e77d4f98588aaa551b068ece403
[WARN] [10-08|22:49:48.918] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:50:00.521] [NewPayload] Handling new payload        height=20923956 hash=0x40d2e0fb121ea3af045bfd5c611d0718ecdc2cd88f6647389aa52832d7a486a4
[INFO] [10-08|22:50:00.521] [EngineBlockDownloader] Downloading PoS headers... hash=0x357e6ff715b7e21faaa4d4ea18593270afd00e77d4f98588aaa551b068ece403 requestId=0
[INFO] [10-08|22:50:30.521] [EngineBlockDownloader] Waiting for headers download to finish
[WARN] [10-08|22:50:32.539] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:50:37.000] [NewPayload] Handling new payload        height=20923959 hash=0xc8179b38691fde4436b004f654d0ff5dcdf07c8ae2d70d55d1ea84fcab2c2237
[INFO] [10-08|22:50:37.000] [EngineBlockDownloader] Downloading PoS headers... hash=0x2a5a02b17b87e5c55164a2ac9319a70cb37610f048e90df532af362128cca20d requestId=0
[WARN] [10-08|22:51:01.938] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:51:12.808] [NewPayload] Handling new payload        height=20923962 hash=0xf1e871d352b01986e0557498d7a43f4ef9b33c714b352869b9db44b61ef9b749
[INFO] [10-08|22:51:12.808] [EngineBlockDownloader] Downloading PoS headers... hash=0x76a5c00e9806a039487fc14b73278edb16dc15548f9ed91aeaa966ea64cd272c requestId=0
[WARN] [10-08|22:51:39.785] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:51:49.079] [NewPayload] Handling new payload        height=20923965 hash=0x306df0f6c8710a08986ebba2325067d373f8b175b0f4d261c8b6842d2432f95e
[INFO] [10-08|22:51:49.079] [EngineBlockDownloader] Downloading PoS headers... hash=0x70237898265ce596987ba097639a8b74fac815c35151c9e87f42c83369f79ad4 requestId=0
[WARN] [10-08|22:52:17.668] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:52:19.249] [p2p] GoodPeers                          eth68=22 eth67=9
[INFO] [10-08|22:52:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.5GB sys=12.5GB
[INFO] [10-08|22:52:19.347] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.5GB sys=12.5GB
[INFO] [10-08|22:52:25.365] [NewPayload] Handling new payload        height=20923968 hash=0x8e68a3b3fa340b698fab5a3cb1aca7deaf097a80cd577335df494066f893f77a
[INFO] [10-08|22:52:25.365] [EngineBlockDownloader] Downloading PoS headers... hash=0x6342b355fcb9691aeece5731e6ac73d52a9bbac48a82793a75c9d2a060ca962e requestId=0
[WARN] [10-08|22:52:55.864] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:53:01.426] [NewPayload] Handling new payload        height=20923971 hash=0x3444121b3689a6df55823b67b54eb1f12f8fead82f29f78d145578ee0c60b6ea
[INFO] [10-08|22:53:01.426] [EngineBlockDownloader] Downloading PoS headers... hash=0x56beccc26fd247b94314cf2de3913e39c74d2561ad94126bdad2c4764f89a4b8 requestId=0
[WARN] [10-08|22:53:27.174] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:53:29.655] [NewPayload] Handling new payload        height=20923973 hash=0xb23b004e7326d4e83870d70cbd96f5c904770fbe27dd7263be86b5697f0ca401
[INFO] [10-08|22:53:29.655] [EngineBlockDownloader] Downloading PoS headers... hash=0x7c9c40acce304ab5dacd8fbeb8679c4f097a173a239847306dc6947f8ec6be0f requestId=0
[WARN] [10-08|22:53:55.010] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:54:00.828] [NewPayload] Handling new payload        height=20923975 hash=0xde93b034bc4fddfb669be9a0615ab6f6cd4707498af589153aabf06146562828
[INFO] [10-08|22:54:00.829] [EngineBlockDownloader] Downloading PoS headers... hash=0xc22c5c26b79544eb0e63f19fd42532e9677664641d836590bb9ce285ba73f2a7 requestId=0
[WARN] [10-08|22:54:29.492] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:54:38.188] [NewPayload] Handling new payload        height=20923978 hash=0x3338b2e970a4a97150e5a116a883632b875c805e8411847f9d570e0f133391c6
[INFO] [10-08|22:54:38.188] [EngineBlockDownloader] Downloading PoS headers... hash=0x5ba08cd86e8c3749ec02aeedeb32db7af07b0e1f9ab99101e3ef7a5590f311d7 requestId=0
[WARN] [10-08|22:55:05.369] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 20882999"
[INFO] [10-08|22:55:12.699] [NewPayload] Handling new payload        height=20923981 hash=0x0008f43c2c491c913147cbc942bad57bdb0a01b13019ea1e3663c5c317609a5c
[INFO] [10-08|22:55:12.699] [EngineBlockDownloader] Downloading PoS headers... hash=0x2bd4a6237986a09aaf40c1ea1870a9fa1c5c04fbfc64dc07c8c8e2393abaca0f requestId=0
[INFO] [10-08|22:55:19.248] [p2p] GoodPeers                          eth68=25 eth67=9
[INFO] [10-08|22:55:19.347] [txpool] stat                            pending=0 baseFee=0 queued=0 alloc=7.3GB sys=12.5GB
[INFO] [10-08|22:55:19.363] [mem] memory stats                       Rss=13.3GB Size=0B Pss=13.3GB SharedClean=12.0KB SharedDirty=0B PrivateClean=3.6GB PrivateDirty=9.7GB Referenced=10.2GB Anonymous=9.7GB Swap=0B alloc=7.3GB sys=12.5GB

Oct 08 '24 22:10 Wazzymandias

Piggybacking @Wazzymandias logs in DBUG:

Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.609] [p2p] Server                             protocol=68 peers=1 trusted=0 inbound=0 closed by remote=39 i/o timeout=28 too many peers=77 EOF=69
Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.609] [p2p] Dial scheduler                     protocol=68 peers=1/33 tried=323 static=0 connect: connection refused=13 i/o timeout=52 connect: no route to host=3
Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.610] [p2p] Discovery table                    protocol=68 version=v4 len=151 live=5 unsol=40 ips=182 db=0 reval=10 RPC timeout=4 unsolicited reply=4 unknown node=2
Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.634] [p2p] Discovery table                    protocol=67 version=v4 len=144 live=5 unsol=20 ips=176 db=0 reval=10 RPC timeout=4 invalid IP in response record: loopback address from non-loopback host=1 unsolicited reply=3 unknown node=2
Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.634] [p2p] Server                             protocol=67 peers=0 trusted=0 inbound=0 closed by remote=25 i/o timeout=37 EOF=75 too many peers=66
Oct 09 00:27:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:43.634] [p2p] Dial scheduler                     protocol=67 peers=0/33 tried=313 static=0 connect: connection refused=22 i/o timeout=57 connect: no route to host=1
Oct 09 00:27:46 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:46.355] [txpool] Commit                          written_kb=96 in=2.332474ms
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.795] [NewPayload] sending block               height=2496737 hash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:27:50 holesky-erigon erigon[78638]: [INFO] [10-09|00:27:50.795] [NewPayload] Handling new payload        height=2496737 hash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.795] [NewPayload] New payload: need to download parent height=2496737 hash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077 parentHash=0x330fa8dbfbe7553c3961caefd51c61c1a63da75d3a3c9be6aa72ef328081fd57
Oct 09 00:27:50 holesky-erigon erigon[78638]: [INFO] [10-09|00:27:50.795] [EngineBlockDownloader] Downloading PoS headers... hash=0x330fa8dbfbe7553c3961caefd51c61c1a63da75d3a3c9be6aa72ef328081fd57 requestId=0
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.795] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.797] [downloader] Request header              number=0 length=192
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.797] [downloader] Sent request                height=0
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.870] [downloader] Collecting...               from=2496736 to=2496545 len=192
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.870] [downloader] Synced                      requestId=0
Oct 09 00:27:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:50.885] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:27:50 holesky-erigon erigon[78638]: [WARN] [10-09|00:27:50.887] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:27:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:51.856] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [ForkChoiceUpdated] sending forkChoiceMessage head=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [ForkChoiceUpdated] Handling fork choice headerHash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [ForkChoiceUpdated] Fork choice: need to download header with hash 7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:27:52 holesky-erigon erigon[78638]: [INFO] [10-09|00:27:52.283] [EngineBlockDownloader] Downloading PoS headers... hash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077 requestId=0
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [downloader] Request header              number=0 length=192
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.283] [downloader] Sent request                height=0
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.354] [downloader] Collecting...               from=2496737 to=2496546 len=192
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.354] [downloader] Synced                      requestId=0
Oct 09 00:27:52 holesky-erigon erigon[78638]: [DBUG] [10-09|00:27:52.376] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:27:52 holesky-erigon erigon[78638]: [WARN] [10-09|00:27:52.385] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.491] [NewPayload] sending block               height=2496738 hash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:01 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:01.491] [NewPayload] Handling new payload        height=2496738 hash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.491] [NewPayload] New payload: need to download parent height=2496738 hash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7 parentHash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077
Oct 09 00:28:01 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:01.491] [EngineBlockDownloader] Downloading PoS headers... hash=0x7cfce18051867d028fd6c92197025918e6e322d94e841c43f750b39af851f077 requestId=0
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.491] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.492] [downloader] Request header              number=0 length=192
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.492] [downloader] Sent request                height=0
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.561] [downloader] Collecting...               from=2496737 to=2496546 len=192
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.562] [downloader] Synced                      requestId=0
Oct 09 00:28:01 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:01.570] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:01 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:01.572] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:02.069] [txpool] Commit                          written_kb=0 in=2.593601ms
Oct 09 00:28:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:02.555] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.025] [ForkChoiceUpdated] sending forkChoiceMessage head=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.025] [ForkChoiceUpdated] Handling fork choice headerHash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.025] [ForkChoiceUpdated] Fork choice: need to download header with hash 93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.025] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:03 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:03.025] [EngineBlockDownloader] Downloading PoS headers... hash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7 requestId=0
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.026] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.028] [downloader] Request header              number=0 length=192
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.028] [downloader] Sent request                height=0
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.099] [downloader] Collecting...               from=2496738 to=2496547 len=192
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.099] [downloader] Synced                      requestId=0
Oct 09 00:28:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:03.123] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:03 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:03.132] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.401] [NewPayload] sending block               height=2496739 hash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:14 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:14.401] [NewPayload] Handling new payload        height=2496739 hash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.401] [NewPayload] New payload: need to download parent height=2496739 hash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff parentHash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7
Oct 09 00:28:14 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:14.401] [EngineBlockDownloader] Downloading PoS headers... hash=0x93da44b2982f15c422cb17f53561f6c8b3630c1f6972bc5918d94b08eb742cf7 requestId=0
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.401] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.401] [downloader] Request header              number=0 length=192
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.401] [downloader] Sent request                height=0
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.471] [downloader] Collecting...               from=2496738 to=2496547 len=192
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.471] [downloader] Synced                      requestId=0
Oct 09 00:28:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:14.480] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:14 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:14.484] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.467] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.706] [ForkChoiceUpdated] sending forkChoiceMessage head=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.706] [ForkChoiceUpdated] Handling fork choice headerHash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.706] [ForkChoiceUpdated] Fork choice: need to download header with hash 2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.706] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:15 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:15.706] [EngineBlockDownloader] Downloading PoS headers... hash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff requestId=0
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.707] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.708] [downloader] Request header              number=0 length=192
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.708] [downloader] Sent request                height=0
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.779] [downloader] Collecting...               from=2496739 to=2496548 len=192
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.781] [downloader] Synced                      requestId=0
Oct 09 00:28:15 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:15.793] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:15 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:15.796] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:17 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:17.790] [txpool] Commit                          written_kb=80 in=9.408155ms
Oct 09 00:28:23 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:23.436] [downloader] Collecting...               from=2496739 to=2496739 len=1
Oct 09 00:28:23 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:23.436] [downloader] posAnchor is nil
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.646] [NewPayload] sending block               height=2496740 hash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:26 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:26.646] [NewPayload] Handling new payload        height=2496740 hash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.647] [NewPayload] New payload: need to download parent height=2496740 hash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435 parentHash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff
Oct 09 00:28:26 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:26.647] [EngineBlockDownloader] Downloading PoS headers... hash=0x2643f86269f479fe497009f7d832f9d0aa695289260bf4f47345e131e50224ff requestId=0
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.647] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.647] [downloader] Request header              number=0 length=192
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.648] [downloader] Sent request                height=0
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.716] [downloader] Collecting...               from=2496739 to=2496548 len=192
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.717] [downloader] Synced                      requestId=0
Oct 09 00:28:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:26.723] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:26 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:26.725] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.715] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [ForkChoiceUpdated] sending forkChoiceMessage head=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [ForkChoiceUpdated] Handling fork choice headerHash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [ForkChoiceUpdated] Fork choice: need to download header with hash 66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:27 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:27.912] [EngineBlockDownloader] Downloading PoS headers... hash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435 requestId=0
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [downloader] Request header              number=0 length=192
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.912] [downloader] Sent request                height=0
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.982] [downloader] Collecting...               from=2496740 to=2496549 len=192
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.983] [downloader] Synced                      requestId=0
Oct 09 00:28:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:27.996] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:28 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:28.005] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:33 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:33.522] [txpool] Commit                          written_kb=1696 in=29.684861ms
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.766] [NewPayload] sending block               height=2496741 hash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:37 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:37.766] [NewPayload] Handling new payload        height=2496741 hash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.766] [NewPayload] New payload: need to download parent height=2496741 hash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31 parentHash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435
Oct 09 00:28:37 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:37.766] [EngineBlockDownloader] Downloading PoS headers... hash=0x66a353b9f683d21c621a701e1538a7178e3c73e17047c892e401707eeaa38435 requestId=0
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.766] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.768] [downloader] Request header              number=0 length=192
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.768] [downloader] Sent request                height=0
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.837] [downloader] Collecting...               from=2496740 to=2496549 len=192
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.838] [downloader] Synced                      requestId=0
Oct 09 00:28:37 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:37.846] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:37 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:37.848] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:38 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:38.832] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.044] [ForkChoiceUpdated] sending forkChoiceMessage head=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.044] [ForkChoiceUpdated] Handling fork choice headerHash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.044] [ForkChoiceUpdated] Fork choice: need to download header with hash 8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.044] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:39 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:39.044] [EngineBlockDownloader] Downloading PoS headers... hash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31 requestId=0
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.045] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.045] [downloader] Request header              number=0 length=192
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.045] [downloader] Sent request                height=0
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.118] [downloader] Collecting...               from=2496741 to=2496550 len=192
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.119] [downloader] Synced                      requestId=0
Oct 09 00:28:39 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:39.132] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:39 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:39.134] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.608] [p2p] Discovery table                    protocol=68 version=v4 len=151 live=13 unsol=40 ips=182 db=0 reval=21 RPC timeout=5 unsolicited reply=4 unknown node=2
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.610] [p2p] Dial scheduler                     protocol=68 peers=1/33 tried=565 static=0 connect: connection refused=24 i/o timeout=78 connect: no route to host=6
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.610] [p2p] Server                             protocol=68 peers=1 trusted=0 inbound=0 too many peers=130 EOF=138 closed by remote=51 i/o timeout=76
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.633] [p2p] Discovery table                    protocol=67 version=v4 len=168 live=10 unsol=20 ips=216 db=0 reval=21 RPC timeout=8 invalid IP in response record: loopback address from non-loopback host=1 unknown node=2 unsolicited reply=3
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.634] [p2p] Server                             protocol=67 peers=1 trusted=0 inbound=0 EOF=121 too many peers=139 closed by remote=48 i/o timeout=72
Oct 09 00:28:43 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:43.634] [p2p] Dial scheduler                     protocol=67 peers=1/33 tried=587 static=0 connect: connection refused=32 i/o timeout=116 connect: no route to host=3
Oct 09 00:28:49 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:49.215] [txpool] Commit                          written_kb=96 in=9.089735ms
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.424] [NewPayload] sending block               height=2496742 hash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:28:50 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:50.424] [NewPayload] Handling new payload        height=2496742 hash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.424] [NewPayload] New payload: need to download parent height=2496742 hash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29 parentHash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31
Oct 09 00:28:50 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:50.425] [EngineBlockDownloader] Downloading PoS headers... hash=0x8374d9935528a69a7c3a22ca7ac6157f0b5e9a1646a6b99c2f80a201c0884d31 requestId=0
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.425] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.426] [downloader] Request header              number=0 length=192
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.426] [downloader] Sent request                height=0
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.496] [downloader] Collecting...               from=2496741 to=2496550 len=192
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.497] [downloader] Synced                      requestId=0
Oct 09 00:28:50 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:50.513] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:50 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:50.515] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.491] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.684] [ForkChoiceUpdated] sending forkChoiceMessage head=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.684] [ForkChoiceUpdated] Handling fork choice headerHash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.684] [ForkChoiceUpdated] Fork choice: need to download header with hash 8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.684] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:28:51 holesky-erigon erigon[78638]: [INFO] [10-09|00:28:51.684] [EngineBlockDownloader] Downloading PoS headers... hash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29 requestId=0
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.684] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.686] [downloader] Request header              number=0 length=192
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.686] [downloader] Sent request                height=0
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.757] [downloader] Collecting...               from=2496742 to=2496551 len=192
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.757] [downloader] Synced                      requestId=0
Oct 09 00:28:51 holesky-erigon erigon[78638]: [DBUG] [10-09|00:28:51.772] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:28:51 holesky-erigon erigon[78638]: [WARN] [10-09|00:28:51.774] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.392] [NewPayload] sending block               height=2496743 hash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:02 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:02.392] [NewPayload] Handling new payload        height=2496743 hash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.392] [NewPayload] New payload: need to download parent height=2496743 hash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236 parentHash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29
Oct 09 00:29:02 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:02.392] [EngineBlockDownloader] Downloading PoS headers... hash=0x8f8bae9a5e535ea757e006887b38c5269cb4bb81dfdee2afa3b14bea42dbab29 requestId=0
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.392] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.393] [downloader] Request header              number=0 length=192
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.393] [downloader] Sent request                height=0
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.462] [downloader] Collecting...               from=2496742 to=2496551 len=192
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.463] [downloader] Synced                      requestId=0
Oct 09 00:29:02 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:02.470] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:02 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:02.472] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.460] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.673] [ForkChoiceUpdated] sending forkChoiceMessage head=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.673] [ForkChoiceUpdated] Handling fork choice headerHash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.673] [ForkChoiceUpdated] Fork choice: need to download header with hash fdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.673] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:03 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:03.674] [EngineBlockDownloader] Downloading PoS headers... hash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236 requestId=0
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.674] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.675] [downloader] Request header              number=0 length=192
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.675] [downloader] Sent request                height=0
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.747] [downloader] Collecting...               from=2496743 to=2496552 len=192
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.749] [downloader] Synced                      requestId=0
Oct 09 00:29:03 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:03.768] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:03 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:03.775] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:04 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:04.921] [txpool] Commit                          written_kb=64 in=1.590542ms
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.066] [NewPayload] sending block               height=2496744 hash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:13 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:13.066] [NewPayload] Handling new payload        height=2496744 hash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.067] [NewPayload] New payload: need to download parent height=2496744 hash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd parentHash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236
Oct 09 00:29:13 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:13.067] [EngineBlockDownloader] Downloading PoS headers... hash=0xfdf046c585e8f2c037ab41a52a9c16294d73a929daf742a9d8c9bca84b26d236 requestId=0
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.067] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.068] [downloader] Request header              number=0 length=192
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.068] [downloader] Sent request                height=0
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.140] [downloader] Collecting...               from=2496743 to=2496552 len=192
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.141] [downloader] Synced                      requestId=0
Oct 09 00:29:13 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:13.155] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:13 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:13.158] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.124] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.369] [ForkChoiceUpdated] sending forkChoiceMessage head=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.369] [ForkChoiceUpdated] Handling fork choice headerHash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.369] [ForkChoiceUpdated] Fork choice: need to download header with hash 2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.369] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:14 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:14.369] [EngineBlockDownloader] Downloading PoS headers... hash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd requestId=0
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.369] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.370] [downloader] Request header              number=0 length=192
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.370] [downloader] Sent request                height=0
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.441] [downloader] Collecting...               from=2496744 to=2496553 len=192
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.442] [downloader] Synced                      requestId=0
Oct 09 00:29:14 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:14.450] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:14 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:14.452] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:20 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:20.635] [txpool] Commit                          written_kb=192 in=2.880758ms
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.764] [NewPayload] sending block               height=2496745 hash=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef
Oct 09 00:29:25 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:25.764] [NewPayload] Handling new payload        height=2496745 hash=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.764] [NewPayload] New payload: need to download parent height=2496745 hash=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef parentHash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd
Oct 09 00:29:25 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:25.764] [EngineBlockDownloader] Downloading PoS headers... hash=0x2fac4d333718228b8f09fd4c62bb90e714f19367e40db1b18027caf929e865cd requestId=0
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.764] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.766] [downloader] Request header              number=0 length=192
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.766] [downloader] Sent request                height=0
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.836] [downloader] Collecting...               from=2496744 to=2496553 len=192
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.837] [downloader] Synced                      requestId=0
Oct 09 00:29:25 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:25.852] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:25 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:25.854] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:29:26 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:26.837] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.032] [ForkChoiceUpdated] sending forkChoiceMessage head=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.032] [ForkChoiceUpdated] Handling fork choice headerHash=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.032] [ForkChoiceUpdated] Fork choice: need to download header with hash c5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.032] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:29:27 holesky-erigon erigon[78638]: [INFO] [10-09|00:29:27.032] [EngineBlockDownloader] Downloading PoS headers... hash=0xc5081ada86b241115ec63ad4783e2cfdb38758337817946b5e5aaa8a02c2afef requestId=0
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.032] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.033] [downloader] Request header              number=0 length=192
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.033] [downloader] Sent request                height=0
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.103] [downloader] Collecting...               from=2496745 to=2496554 len=192
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.104] [downloader] Synced                      requestId=0
Oct 09 00:29:27 holesky-erigon erigon[78638]: [DBUG] [10-09|00:29:27.121] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:29:27 holesky-erigon erigon[78638]: [WARN] [10-09|00:29:27.123] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"

Oct 09 '24 05:10 86b

Logs with a reboot:

Oct 09 00:35:07 holesky-erigon systemd[1]: Started Erigon.
Oct 09 00:35:07 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:07.792] logging to file system                   log dir=/data/.erigon/logs file prefix=erigon log level=info json=false
Oct 09 00:35:07 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:07.793] Enabling metrics export to prometheus    path=http://0.0.0.0:6060/debug/metrics/prometheus
Oct 09 00:35:07 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:07.793] Build info                               git_branch=main git_tag=v3.0.0-alpha4-39-g8ef8ca2 git_commit=8ef8ca2a3ca31e5f96161b1fd2b8dcc3e640159c
Oct 09 00:35:07 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:07.794]
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ########b          oo                               d####b.
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ##                                                      '##
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ##aaaa    ##d###b. dP .d####b. .d####b. ##d###b.     aaad#'
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ##        ##'  '## ## ##'  '## ##'  '## ##'  '##        '##
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ##        ##       ## ##.  .## ##.  .## ##    ##        .##
Oct 09 00:35:07 holesky-erigon erigon[78701]:         ########P dP       dP '####P## '#####P' dP    dP    d#####P
Oct 09 00:35:07 holesky-erigon erigon[78701]:                                    .##
Oct 09 00:35:07 holesky-erigon erigon[78701]:                                d####P
Oct 09 00:35:07 holesky-erigon erigon[78701]:
Oct 09 00:35:07 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:07.794] Starting Erigon on Holesky testnet...
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.489] Maximum peer count                       ETH=100 total=100
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.489] starting HTTP APIs                       port=8545 APIs=net,web3,eth,admin,debug,txpool,engine,rpc,trace
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.489] torrent verbosity                        level=WRN
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.491] [torrent] Public IP                      ip=67.108.29.4
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.493] Set global gas cap                       cap=5000000000
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.496] [Downloader] Running with                ipv6-enabled=false ipv4-enabled=true download.rate=2500mb upload.rate=4mb
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.497] Opening Database                         label=chaindata path=/data/.erigon/chaindata
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.508] [db] open                                label=chaindata sizeLimit=8TB pageSize=4096
Oct 09 00:35:08 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:08.511] Initialised chain configuration          config="{ChainID: 17000, Homestead: 0, DAO: <nil>, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 0, Muir Glacier: <nil>, Berlin: 0, London: 0, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: 0, Merge Netsplit: <nil>, Shanghai: 1696000704, Cancun: 1707305664, Prague: <nil>, Osaka: <nil>, Engine: unknown}" genesis=0xb5f7f912443c940f21fd611f12828d75b534364ed9e95ca4e307729a4661bde4
Oct 09 00:35:08 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:08.587] [agg] skip garbage file                  name=v1-receipt.40-41.ef
Oct 09 00:35:11 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:11.104] [db] open                                label=downloader sizeLimit=16GB pageSize=4096
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.832] Initialising Ethereum protocol           network=17000
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.832] Disk storage enabled for ethash DAGs     dir=/data/.erigon/ethash-dags count=2
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.832] [snapshots.webseed] providers            http=7 disk=0
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.861] [db] open                                label=txpool sizeLimit=1TB pageSize=16384
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.862] Starting private RPC server              on=127.0.0.1:9090
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.862] new subscription to logs established
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.862] rpc filters: subscribing to Erigon events
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.862] Establishing event subscription channel with the RPC daemon ...
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.863] new subscription to newHeaders established
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.863] [txpool] Started
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.863] New txs subscriber joined
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.864] Reading JWT secret                       path=/data/jwt/jwtsecret
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.864] HTTP endpoint opened for Engine API      url=[::]:8551 ws=true ws.compression=true
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.865] JsonRpc endpoint opened                  ws=true ws.compression=true grpc=false websocket.url=[::]:8546 http.url=[::]:8545
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.868] [db] open                                label=diagnostics sizeLimit=16GB pageSize=4096
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.908] [db] open                                label=sentry sizeLimit=8GB pageSize=4096
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.911] [OtterSync] Starting Ottersync
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.912]
Oct 09 00:35:31 holesky-erigon erigon[78701]:    _____ _             _   _                ____  _   _
Oct 09 00:35:31 holesky-erigon erigon[78701]:   / ____| |           | | (_)              / __ \| | | |
Oct 09 00:35:31 holesky-erigon erigon[78701]:  | (___ | |_ __ _ _ __| |_ _ _ __   __ _  | |  | | |_| |_ ___ _ __ ___ _   _ _ __   ___
Oct 09 00:35:31 holesky-erigon erigon[78701]:   \___ \| __/ _ | '__| __| | '_ \ / _ | | |  | | __| __/ _ \ '__/ __| | | | '_ \ / __|
Oct 09 00:35:31 holesky-erigon erigon[78701]:   ____) | || (_| | |  | |_| | | | | (_| | | |__| | |_| ||  __/ |  \__ \ |_| | | | | (__ _ _ _
Oct 09 00:35:31 holesky-erigon erigon[78701]:  |_____/ \__\__,_|_|   \__|_|_| |_|\__, |  \____/ \__|\__\___|_|  |___/\__, |_| |_|\___(_|_|_)
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                     __/ |                               __/ |
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                    |___/                               |___/
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                         .:-===++**++===-:
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                    :=##%@@@@@@@@@@@@@@@@@@%#*=.
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                .=#@@@@@@%##+====--====+##@@@@@@@#=.     ...
Oct 09 00:35:31 holesky-erigon erigon[78701]:                    .=**###*=:+#@@@@%*=:.                  .:=#%@@@@#==#@@@@@%#-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                  -#@@@@%%@@@@@@%+-.                            .=*%@@@@#*+*#@@@%=
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 =@@@*:    -%%+:                                    -#@+.     =@@@-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 %@@#     +@#.                                        :%%-     %@@*
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 @@@+    +%=.     -+=                        :=-       .#@-    %@@#
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 *@@%:  #@-      =@@@*                      +@@@%.       =@= -*@@@:
Oct 09 00:35:31 holesky-erigon erigon[78701]:                  #@@@##@+       #@@@@.                     %@@@@=        #@%@@@#-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                   :#@@@@:       +@@@#       :=++++==-.     *@@@@:        =@@@@-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                   =%@@%=         +#*.    =#%#+==-==+#%%=:  .+#*:         .#@@@#.
Oct 09 00:35:31 holesky-erigon erigon[78701]:                  +@@%+.               .+%+-.          :=##-                :#@@@-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 -@@@=                -%#:     ..::.      +@*                 +@@%.
Oct 09 00:35:31 holesky-erigon erigon[78701]:     .::-========*@@@..              -@#      +%@@@@%.     -@#               .-@@@+=======-
Oct 09 00:35:31 holesky-erigon erigon[78701]: .:-====----:::::#@@%:--=::::..      #@:      *@@@@@%:      *@=      ..:-:-=--:@@@+::::----
Oct 09 00:35:31 holesky-erigon erigon[78701]:                 =@@@:.......        @@        :+@#=.       -@+        .......-@@@:
Oct 09 00:35:31 holesky-erigon erigon[78701]:        .:=++####*%@@%=--::::..      @@   %#     %*    :@*  -@+      ...::---+@@@#*#*##+=-:
Oct 09 00:35:31 holesky-erigon erigon[78701]:   ..--==::..     :%@@@-   ..:::..   @@   +@*:.-#@@+-.-#@-  -@+   ..:::..  .+@@@#.     ..:-
Oct 09 00:35:31 holesky-erigon erigon[78701]:                   .#@@@##-:.        @@    :+#@%=.:+@@#=.   -@+        .-=#@@@@+
Oct 09 00:35:31 holesky-erigon erigon[78701]:              -=+++=--+%@@%+=.       @@       +%*=+#%-      -@+       :=#@@@%+--++++=:
Oct 09 00:35:31 holesky-erigon erigon[78701]:          .=**=:.      .=*@@@@@#=:.  @@         :--.        -@+  .-+#@@@@%+:       .:=*+-.
Oct 09 00:35:31 holesky-erigon erigon[78701]:         ::.              .=*@@@@@@%#@@+=-:..         ..::=+#@%#@@@@@@%+-.             ..-.
Oct 09 00:35:31 holesky-erigon erigon[78701]:                             ..=*#@@@@@@@@@@@@@@@%%@@@@@@@@@@@@@@%#+-.
Oct 09 00:35:31 holesky-erigon erigon[78701]:                                   .:-==++*#######%######**+==-:
Oct 09 00:35:31 holesky-erigon erigon[78701]:
Oct 09 00:35:31 holesky-erigon erigon[78701]:
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.912] [1/6 OtterSync] Requesting downloads
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.919] [db] open                                label=sentry sizeLimit=8GB pageSize=4096
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.956] QuerySeeds read nodes from the node DB   count=0
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.957] QuerySeeds read nodes from the node DB   count=0
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.957] Started P2P networking                   version=68 self=enode://271e8452dc41e9c7c5bc132d3a725d8ddcfa274aea4135b6873166feeb1c13edd7aa7b9b7bbb4cee29e2a64627848fb89ece672bea97df55c7a6d49a9715b58f@68.72.23.1:50169 name=erigon/v3.00.0-alpha4-8ef8ca2a/linux-amd64/go1.22.7
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.969] QuerySeeds read nodes from the node DB   count=0
Oct 09 00:35:31 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:31.969] Started P2P networking                   version=67 self=enode://271e8452dc41e9c7c5bc132d3a725d8ddcfa274aea4135b6873166feeb1c13edd7aa7b9b7bbb4cee29e2a64627848fb89ece672bea97df55c7a6d49a9715b58f@68.72.23.1:50170 name=erigon/v3.00.0-alpha4-8ef8ca2a/linux-amd64/go1.22.7
Oct 09 00:35:31 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:31.970] QuerySeeds read nodes from the node DB   count=0
Oct 09 00:35:32 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:32.403] [agg] skip garbage file                  name=v1-receipt.40-41.ef
Oct 09 00:35:32 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:32.429] [snapshots.webseed] /manifest.txt retrieval failed, no downloads from this webseed webseed=https://caplin-snapshots-holesky.erigon.network status="404 Not Found"
Oct 09 00:35:32 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:32.430] [snapshots.webseed] get from HTTP provider err="webseed.http: status=404, url=https://caplin-snapshots-holesky.erigon.network/manifest.txt" url=https://caplin-snapshots-holesky.erigon.network
Oct 09 00:35:32 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:32.607] [1/6 OtterSync] Requesting downloads
Oct 09 00:35:32 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:32.989] [snapshots.webseed] /manifest.txt retrieval failed, no downloads from this webseed webseed=https://erigon2-v1-snapshots-holesky.erigon.network status="404 Not Found"
Oct 09 00:35:32 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:32.989] [snapshots.webseed] get from HTTP provider err="webseed.http: status=404, url=https://erigon2-v1-snapshots-holesky.erigon.network/manifest.txt" url=https://erigon2-v1-snapshots-holesky.erigon.network
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.114] [agg] skip garbage file                  name=v1-receipt.40-41.ef
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.147] [agg] skip garbage file                  name=v1-receipt.40-41.ef
Oct 09 00:35:33 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:33.176] [snapshots:download] Stat                blocks=2.49M indices=2.49M alloc=1.1GB sys=1.4GB
Oct 09 00:35:33 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:33.179] [snapshots:history] Stat                 blocks=2.45M txs=95.31M txNum2blockNum="32=1426K,48=1841K,56=2162K,60=2398K,61=2458K" first_history_idx_in_db=2458964 last_comitment_block=2458964 last_comitment_tx_num=95312500 alloc=1.1GB sys=1.4GB
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.179] [1/6 OtterSync] DONE                     in=1.267961948s
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.179] [2/6 BlockHashes] DONE                   in=42.023µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.179] [3/6 Senders] DONE                       in=258.789µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.181] [4/6 Execution] DONE                     in=1.445888ms
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.181] [5/6 TxLookup] DONE                      in=470.709µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.181] [6/6 Finish] DONE                        in=33.075µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.181] [6/6 Finish] Prune done                  in=44.145µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.181] [5/6 TxLookup] Prune done                in=37.371µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.182] [4/6 Execution] Prune done               in=345.584µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.182] [3/6 Senders] Prune done                 in=32.494µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.182] [2/6 BlockHashes] Prune done             in=40.228µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.183] [snapshots] Prune Blocks                 to=2494976 limit=10000
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.183] [1/6 OtterSync] Prune done               in=788.105µs
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.377] [snapshots.webseed] /manifest.txt retrieval failed, no downloads from this webseed webseed=https://erigon2-v2-snapshots-holesky.erigon.network status="404 Not Found"
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.378] [snapshots.webseed] get from HTTP provider err="webseed.http: status=404, url=https://erigon2-v2-snapshots-holesky.erigon.network/manifest.txt" url=https://erigon2-v2-snapshots-holesky.erigon.network
Oct 09 00:35:33 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:33.412] request failed                           app=downloader error="Head \"https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon2-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:34 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:34.490] request failed                           app=downloader error="Head \"https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon2-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:36 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:36.193] [downloader] Collecting...               from=2496773 to=2496773 len=1
Oct 09 00:35:36 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:36.193] [downloader] posAnchor is nil
Oct 09 00:35:36 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:36.501] request failed                           app=downloader error="Head \"https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon2-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.921] ExchangeCapabilities mismatches          cl_unsupported="[engine_exchangeTransitionConfigurationV1 engine_getPayloadBodiesByHashV2 engine_getPayloadBodiesByRangeV2]" erigon_unsupported=[engine_getClientVersionV1]
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.924] [ForkChoiceUpdated] sending forkChoiceMessage head=0x20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.924] [ForkChoiceUpdated] Handling fork choice headerHash=0x20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.925] [ForkChoiceUpdated] Fork choice: need to download header with hash 20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.925] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:37 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:37.925] [EngineBlockDownloader] Downloading PoS headers... hash=0x20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe requestId=0
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.925] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.927] [downloader] Request header              number=0 length=192
Oct 09 00:35:37 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:37.927] [downloader] Sent request                height=0
Oct 09 00:35:38 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:38.062] [downloader] Collecting...               from=2496771 to=2496580 len=192
Oct 09 00:35:38 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:38.087] [downloader] Synced                      requestId=0
Oct 09 00:35:38 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:38.099] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:38 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:38.105] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:40 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:40.511] request failed                           app=downloader error="Head \"https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon2-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:41 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:41.240] [downloader] Collecting...               from=2496773 to=2496773 len=1
Oct 09 00:35:41 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:41.240] [downloader] posAnchor is nil
Oct 09 00:35:43 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:43.378] [snapshots.webseed] get from HTTP provider err="webseed.http: make request: Head \"https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt\": context deadline exceeded, url=https://erigon2-v3-snapshots-holesky.erigon.network/manifest.txt" url=https://erigon2-v3-snapshots-holesky.erigon.network
Oct 09 00:35:43 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:43.803] [snapshots.webseed] get from HTTP provider manifest-len=1839 url=https://erigon3-v1-snapshots-holesky.erigon.network
Oct 09 00:35:43 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:43.813] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:44 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:44.823] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.411] [NewPayload] sending block               height=2496772 hash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:46 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:46.411] [NewPayload] Handling new payload        height=2496772 hash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.411] [NewPayload] New payload: need to download parent height=2496772 hash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00 parentHash=0x20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe
Oct 09 00:35:46 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:46.411] [EngineBlockDownloader] Downloading PoS headers... hash=0x20a8c45d654df07c15c1f2b5030ae826fca19b51544458171cf1804cf90c91fe requestId=0
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.411] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.411] [downloader] Request header              number=0 length=192
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.412] [downloader] Sent request                height=0
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.566] [downloader] Collecting...               from=2496771 to=2496580 len=192
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.569] [downloader] Synced                      requestId=0
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.578] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:46 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:46.581] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.801] [downloader] Collecting...               from=2496771 to=2496580 len=192
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.801] [downloader] posAnchor is nil
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.902] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:46 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:46.957] [txpool] Commit                          written_kb=1696 in=11.191037ms
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.472] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.742] [ForkChoiceUpdated] sending forkChoiceMessage head=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.742] [ForkChoiceUpdated] Handling fork choice headerHash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.742] [ForkChoiceUpdated] Fork choice: need to download header with hash 9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.742] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:47 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:47.742] [EngineBlockDownloader] Downloading PoS headers... hash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00 requestId=0
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.742] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.744] [downloader] Request header              number=0 length=192
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.744] [downloader] Sent request                height=0
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.898] [downloader] Collecting...               from=2496772 to=2496581 len=192
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.900] [downloader] Synced                      requestId=0
Oct 09 00:35:47 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:47.910] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:47 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:47.913] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.138] [downloader] Collecting...               from=2496772 to=2496581 len=192
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.138] [downloader] posAnchor is nil
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.371] [NewPayload] sending block               height=2496773 hash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:48 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:48.371] [NewPayload] Handling new payload        height=2496773 hash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.371] [NewPayload] New payload: need to download parent height=2496773 hash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1 parentHash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00
Oct 09 00:35:48 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:48.371] [EngineBlockDownloader] Downloading PoS headers... hash=0x9994edd08281eaab2771fd5f8100577d91bdf50187c8bc4379c31973a2510a00 requestId=0
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.371] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.371] [downloader] Request header              number=0 length=192
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.371] [downloader] Sent request                height=0
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.527] [downloader] Collecting...               from=2496772 to=2496581 len=192
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.530] [downloader] Synced                      requestId=0
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.550] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:48 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:48.559] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.634] [downloader] Collecting...               from=2496772 to=2496581 len=192
Oct 09 00:35:48 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:48.634] [downloader] posAnchor is nil
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.459] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.697] [ForkChoiceUpdated] sending forkChoiceMessage head=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.697] [ForkChoiceUpdated] Handling fork choice headerHash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.697] [ForkChoiceUpdated] Fork choice: need to download header with hash b8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.697] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:49 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:49.697] [EngineBlockDownloader] Downloading PoS headers... hash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1 requestId=0
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.697] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.699] [downloader] Request header              number=0 length=192
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.699] [downloader] Sent request                height=0
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.854] [downloader] Collecting...               from=2496773 to=2496582 len=192
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.857] [downloader] Synced                      requestId=0
Oct 09 00:35:49 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:49.866] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:49 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:49.870] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.091] [downloader] Collecting...               from=2496773 to=2496582 len=192
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.091] [downloader] posAnchor is nil
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.423] [NewPayload] sending block               height=2496774 hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:50 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:50.423] [NewPayload] Handling new payload        height=2496774 hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.423] [NewPayload] New payload: need to download parent height=2496774 hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65 parentHash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1
Oct 09 00:35:50 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:50.423] [EngineBlockDownloader] Downloading PoS headers... hash=0xb8da1e77bf72e2c8a7019b2e2ef6e40ccc4cd15418e59516458b0b1b5b9d6bd1 requestId=0
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.423] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.424] [downloader] Request header              number=0 length=192
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.424] [downloader] Sent request                height=0
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.579] [downloader] Collecting...               from=2496773 to=2496582 len=192
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.581] [downloader] Synced                      requestId=0
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.592] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:50 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:50.595] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.686] [downloader] Collecting...               from=2496773 to=2496582 len=192
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.686] [downloader] posAnchor is nil
Oct 09 00:35:50 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:50.980] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.480] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.720] [ForkChoiceUpdated] sending forkChoiceMessage head=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.720] [ForkChoiceUpdated] Handling fork choice headerHash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.720] [ForkChoiceUpdated] Fork choice: need to download header with hash 67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.720] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:51 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:51.720] [EngineBlockDownloader] Downloading PoS headers... hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65 requestId=0
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.721] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.721] [downloader] Request header              number=0 length=192
Oct 09 00:35:51 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:51.721] [downloader] Sent request                height=0
Oct 09 00:35:51 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:51.832] [1/6 OtterSync] Downloading complete     time=19.920750781s
Oct 09 00:35:52 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:52.115] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:35:52 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:52.118] [downloader] Synced                      requestId=0
Oct 09 00:35:52 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:52.130] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:35:52 holesky-erigon erigon[78701]: [WARN] [10-09|00:35:52.133] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:35:53 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:53.372] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:35:53 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:53.372] [downloader] posAnchor is nil
Oct 09 00:35:53 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:53.803] [snapshots.webseed] get from HTTP provider err="webseed.http: make request: Head \"https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt\": context deadline exceeded, url=https://erigon3-v3-snapshots-holesky.erigon.network/manifest.txt" url=https://erigon3-v3-snapshots-holesky.erigon.network
Oct 09 00:35:53 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:53.813] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt
Oct 09 00:35:54 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:54.823] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt
Oct 09 00:35:56 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:56.837] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.860] [ForkChoiceUpdated] sending forkChoiceMessage head=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.860] [ForkChoiceUpdated] Handling fork choice headerHash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.860] [ForkChoiceUpdated] Fork choice: need to download header with hash 67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.860] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:35:59 holesky-erigon erigon[78701]: [INFO] [10-09|00:35:59.861] [EngineBlockDownloader] Downloading PoS headers... hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65 requestId=0
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.861] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.862] [downloader] Request header              number=0 length=192
Oct 09 00:35:59 holesky-erigon erigon[78701]: [DBUG] [10-09|00:35:59.862] [downloader] Sent request                height=0
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.018] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.021] [downloader] Synced                      requestId=0
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.028] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:00 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:00.032] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.253] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.254] [downloader] posAnchor is nil
Oct 09 00:36:00 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:00.848] request failed                           app=downloader error="Head \"https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt\": dial tcp: lookup erigon3-v3-snapshots-holesky.erigon.network on 127.0.0.53:53: no such host" method=HEAD url=https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt
Oct 09 00:36:02 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:02.033] [txpool] Commit                          written_kb=320 in=3.27914ms
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.310] [NewPayload] sending block               height=2496775 hash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:03 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:03.311] [NewPayload] Handling new payload        height=2496775 hash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.311] [NewPayload] New payload: need to download parent height=2496775 hash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c parentHash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65
Oct 09 00:36:03 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:03.311] [EngineBlockDownloader] Downloading PoS headers... hash=0x67add3ca29bfbb96a3e3054320adcb4d07d98a1404f31786f273642210f5dc65 requestId=0
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.311] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.313] [downloader] Request header              number=0 length=192
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.313] [downloader] Sent request                height=0
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.466] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.469] [downloader] Synced                      requestId=0
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.479] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:03 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:03.482] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.703] [downloader] Collecting...               from=2496774 to=2496583 len=192
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.703] [downloader] posAnchor is nil
Oct 09 00:36:03 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:03.804] [snapshots.webseed] get from HTTP provider err="webseed.http: make request: Head \"https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt\": context deadline exceeded, url=https://erigon3-v3-snapshots-holesky.erigon.network/v2/manifest.txt" url=https://erigon3-v3-snapshots-holesky.erigon.network/v2
Oct 09 00:36:04 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:04.633] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.474] [ForkChoiceUpdated] sending forkChoiceMessage head=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.474] [ForkChoiceUpdated] Handling fork choice headerHash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.474] [ForkChoiceUpdated] Fork choice: need to download header with hash 19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.475] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:12 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:12.475] [EngineBlockDownloader] Downloading PoS headers... hash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c requestId=0
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.475] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.477] [downloader] Request header              number=0 length=192
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.477] [downloader] Sent request                height=0
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.631] [downloader] Collecting...               from=2496775 to=2496584 len=192
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.635] [downloader] Synced                      requestId=0
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.645] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:12 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:12.649] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.868] [downloader] Collecting...               from=2496775 to=2496584 len=192
Oct 09 00:36:12 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:12.868] [downloader] posAnchor is nil
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.556] [NewPayload] sending block               height=2496776 hash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:13 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:13.556] [NewPayload] Handling new payload        height=2496776 hash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.556] [NewPayload] New payload: need to download parent height=2496776 hash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb parentHash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c
Oct 09 00:36:13 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:13.556] [EngineBlockDownloader] Downloading PoS headers... hash=0x19e9f914c98174c10e1190529d8d9840e298778dc3f9347dd5b1e7cfc347094c requestId=0
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.556] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.557] [downloader] Request header              number=0 length=192
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.557] [downloader] Sent request                height=0
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.714] [downloader] Collecting...               from=2496775 to=2496584 len=192
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.717] [downloader] Synced                      requestId=0
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.725] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:13 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:13.728] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.947] [downloader] Collecting...               from=2496775 to=2496584 len=192
Oct 09 00:36:13 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:13.947] [downloader] posAnchor is nil
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.626] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.951] [ForkChoiceUpdated] sending forkChoiceMessage head=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.951] [ForkChoiceUpdated] Handling fork choice headerHash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.951] [ForkChoiceUpdated] Fork choice: need to download header with hash b437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.951] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:14 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:14.952] [EngineBlockDownloader] Downloading PoS headers... hash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb requestId=0
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.952] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.952] [downloader] Request header              number=0 length=192
Oct 09 00:36:14 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:14.952] [downloader] Sent request                height=0
Oct 09 00:36:15 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:15.113] [downloader] Collecting...               from=2496776 to=2496585 len=192
Oct 09 00:36:15 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:15.117] [downloader] Synced                      requestId=0
Oct 09 00:36:15 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:15.344] [downloader] Collecting...               from=2496776 to=2496585 len=192
Oct 09 00:36:15 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:15.345] [downloader] posAnchor is nil
Oct 09 00:36:15 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:15.396] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:15 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:15.399] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:17 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:17.117] [txpool] Commit                          written_kb=448 in=4.149867ms
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.601] [NewPayload] sending block               height=2496777 hash=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9
Oct 09 00:36:27 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:27.601] [NewPayload] Handling new payload        height=2496777 hash=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.602] [NewPayload] New payload: need to download parent height=2496777 hash=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9 parentHash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb
Oct 09 00:36:27 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:27.602] [EngineBlockDownloader] Downloading PoS headers... hash=0xb437316fe48d6ed6c0e556ac611dbc33725f2a01530a71f76f93a62802b12ddb requestId=0
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.602] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.604] [downloader] Request header              number=0 length=192
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.604] [downloader] Sent request                height=0
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.763] [downloader] Collecting...               from=2496776 to=2496585 len=192
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.768] [downloader] Synced                      requestId=0
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.777] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:27 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:27.780] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.995] [downloader] Collecting...               from=2496776 to=2496585 len=192
Oct 09 00:36:27 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:27.996] [downloader] posAnchor is nil
Oct 09 00:36:28 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:28.677] [NewPayload] got reply                   payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.088] [ForkChoiceUpdated] sending forkChoiceMessage head=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.088] [ForkChoiceUpdated] Handling fork choice headerHash=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.089] [ForkChoiceUpdated] Fork choice: need to download header with hash 52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.089] [ForkChoiceUpdated] got reply            payloadStatus="&{Status:SYNCING ValidationError:<nil> LatestValidHash:<nil> CriticalError:<nil>}"
Oct 09 00:36:29 holesky-erigon erigon[78701]: [INFO] [10-09|00:36:29.089] [EngineBlockDownloader] Downloading PoS headers... hash=0x52e6088e6c3aae388d49ceb9f3b788691468cf364ebafc1820f50197502e1ea9 requestId=0
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.089] [downloader] Set posAnchor               blockHeight=1
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.089] [downloader] Request header              number=0 length=192
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.089] [downloader] Sent request                height=0
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.243] [downloader] Collecting...               from=2496777 to=2496586 len=192
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.246] [downloader] Synced                      requestId=0
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.257] [db] open                                label=inMem sizeLimit=2TB pageSize=4096
Oct 09 00:36:29 holesky-erigon erigon[78701]: [WARN] [10-09|00:36:29.261] [EngineBlockDownloader] Could not download bodies err="canonical marker not found: 2496728"
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.481] [downloader] Collecting...               from=2496777 to=2496586 len=192
Oct 09 00:36:29 holesky-erigon erigon[78701]: [DBUG] [10-09|00:36:29.481] [downloader] posAnchor is nil

Oct 09 '24 05:10 86b

same here, I'm running on 3.00.0-alpha4-9c84bf51

The sync gets into a loop and even after restarting erigon I get: Could not download bodies err="canonical marker not found: 20901364"

first error in the log was this segfault on recsplit index lookup:

{"log":"unexpected fault address 0x7ce28e245875\n","stream":"stderr","time":"2024-10-08T13:10:41.09096599Z"}
{"log":"fatal error: fault\n","stream":"stderr","time":"2024-10-08T13:10:41.090985256Z"}
{"log":"[signal SIGSEGV: segmentation violation code=0x1 addr=0x7ce28e245875 pc=0xb7d6f4]\n","stream":"stderr","time":"2024-10-08T13:10:41.094141917Z"}
{"log":"\n","stream":"stderr","time":"2024-10-08T13:10:41.094165822Z"}
{"log":"goroutine 1349 gp=0xc00be7b6c0 m=19 mp=0xc001900008 [running]:\n","stream":"stderr","time":"2024-10-08T13:10:41.094171051Z"}
{"log":"runtime.throw({0x2a6811d?, 0xd?})\n","stream":"stderr","time":"2024-10-08T13:10:41.094517112Z"}
{"log":"\u0009runtime/panic.go:1067 +0x48 fp=0xc26aec5600 sp=0xc26aec55d0 pc=0x494748\n","stream":"stderr","time":"2024-10-08T13:10:41.094529595Z"}
{"log":"runtime.sigpanic()\n","stream":"stderr","time":"2024-10-08T13:10:41.095073447Z"}
{"log":"\u0009runtime/signal_unix.go:914 +0x26c fp=0xc26aec5660 sp=0xc26aec5600 pc=0x4969cc\n","stream":"stderr","time":"2024-10-08T13:10:41.095081302Z"}
{"log":"encoding/binary.bigEndian.Uint64(...)\n","stream":"stderr","time":"2024-10-08T13:10:41.09508592Z"}
{"log":"\u0009encoding/binary/binary.go:188\n","stream":"stderr","time":"2024-10-08T13:10:41.095090178Z"}
{"log":"github.com/erigontech/erigon-lib/recsplit.(*Index).Lookup(0xc15f422600, 0x23e4564f7ee906e1, 0x195dc211d701058f)\n","stream":"stderr","time":"2024-10-08T13:10:41.095094366Z"}
{"log":"\u0009github.com/erigontech/[email protected]/recsplit/index.go:353 +0x7b4 fp=0xc26aec5718 sp=0xc26aec5660 pc=0xb7d6f4\n","stream":"stderr","time":"2024-10-08T13:10:41.095098604Z"}
{"log":"github.com/erigontech/erigon-lib/recsplit.(*IndexReader).Lookup(0xc26aec5828, {0xc0e674fb40?, 0xc26aec57f8?, 0x48ef9d?})\n","stream":"stderr","time":"2024-10-08T13:10:41.095102832Z"}
{"log":"\u0009github.com/erigontech/[email protected]/recsplit/index_reader.go:51 +0x45 fp=0xc26aec5748 sp=0xc26aec5718 pc=0xb7e185\n","stream":"stderr","time":"2024-10-08T13:10:41.09510694Z"}
{"log":"github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks.(*BlockReader).headerFromSnapshotByHash(0x7e232c6d99a0?, {0xc8, 0xd8, 0x1b, 0xb2, 0x7d, 0x21, 0xd9, 0x91, 0x59, ...}, ...)\n","stream":"stderr","time":"2024-10-08T13:10:41.095113372Z"}
{"log":"\u0009github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks/block_reader.go:917 +0x12a fp=0xc26aec5898 sp=0xc26aec5748 pc=0x16ea48a\n","stream":"stderr","time":"2024-10-08T13:10:41.095120165Z"}
{"log":"github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks.(*BlockReader).HeaderByHash(0xc001248cb0, {0x0?, 0x0?}, {0x7e232c6d99a0?, 0xc1861e88c0?}, {0xc8, 0xd8, 0x1b, 0xb2, 0x7d, ...})\n","stream":"stderr","time":"2024-10-08T13:10:41.095723488Z"}
{"log":"\u0009github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks/block_reader.go:532 +0x1a8 fp=0xc26aec5948 sp=0xc26aec5898 pc=0x16e70a8\n","stream":"stderr","time":"2024-10-08T13:10:41.095756821Z"}
{"log":"github.com/erigontech/erigon/turbo/stages/headerdownload.(*HeaderDownload).ProcessHeadersPOS(0xc000ff7600, {0xc013397808, 0xc0, 0x18?}, {0x7e232c6d99a0, 0xc1861e88c0}, {0xf1, 0x24, 0x20, 0xf, ...})\n","stream":"stderr","time":"2024-10-08T13:10:41.095784453Z"}
{"log":"\u0009github.com/erigontech/erigon/turbo/stages/headerdownload/header_algos.go:731 +0x64f fp=0xc26aec5b00 sp=0xc26aec5948 pc=0x1fba78f\n","stream":"stderr","time":"2024-10-08T13:10:41.095802977Z"}
{"log":"github.com/erigontech/erigon/p2p/sentry/sentry_multi_client.(*MultiClient).blockHeaders(0xc0017e0210, {0x32d6798, 0xc00c20b720}, {0xc183464708, 0xc0, 0x45de5c?}, 0xc0a2d0ce00, 0xc1861e8700, {0x32f6900, 0xc00a198138})\n","stream":"stderr","time":"2024-10-08T13:10:41.095819599Z"}
{"log":"\u0009github.com/erigontech/erigon/p2p/sentry/sentry_multi_client/sentry_multi_client.go:350 +0xafb fp=0xc26aec5d30 sp=0xc26aec5b00 pc=0x228ea5b\n","stream":"stderr","time":"2024-10-08T13:10:41.095832453Z"}
{"log":"github.com/erigontech/erigon/p2p/sentry/sentry_multi_client.(*MultiClient).blockHeaders66(0xc0017e0210, {0x32d6798, 0xc00c20b720}, 0xc0138f9bd0, {0x32f6900, 0xc00a198138})\n","stream":"stderr","time":"2024-10-08T13:10:41.095846038Z"}
{"log":"\u0009github.com/erigontech/erigon/p2p/sentry/sentry_multi_client/sentry_multi_client.go:303 +0x29f fp=0xc26aec5dd8 sp=0xc26aec5d30 pc=0x228deff\n","stream":"stderr","time":"2024-10-08T13:10:41.095860295Z"}
{"log":"github.com/erigontech/erigon/p2p/sentry/sentry_multi_client.(*MultiClient).handleInboundMessage(0xffffffffffffffff?, {0x32d6798?, 0xc00c20b720?}, 0x0?, {0x32f6900?, 0xc00a198138?})\n","stream":"stderr","time":"2024-10-08T13:10:41.095887837Z"}
{"log":"\u0009github.com/erigontech/erigon/p2p/sentry/sentry_multi_client/sentry_multi_client.go:667 +0x7c fp=0xc26aec5e28 sp=0xc26aec5dd8 pc=0x2291a7c\n","stream":"stderr","time":"2024-10-08T13:10:41.095898717Z"}
{"log":"github.com/erigontech/erigon/p2p/sentry/sentry_multi_client.(*MultiClient).HandleInboundMessage(0xc0017e0210, {0x32d6798, 0xc00c20b720}, 0xc0138f9bd0, {0x32f6900, 0xc00a198138})\n","stream":"stderr","time":"2024-10-08T13:10:41.095909578Z"}
{"log":"\u0009github.com/erigontech/erigon/p2p/sentry/sentry_multi_client/sentry_multi_client.go:644 +0x91 fp=0xc26aec5ec0 sp=0xc26aec5e28 pc=0x2291611\n","stream":"stderr","time":"2024-10-08T13:10:41.095922041Z"}
{"log":"github.com/erigontech/erigon/p2p/sentry/sentry_multi_client.(*MultiClient).HandleInboundMessage-fm({0x32d6798?, 0xc00c20b720?}, 0x0?, {0x32f6900?, 0xc00a198138?})\n","stream":"stderr","time":"2024-10-08T13:10:41.095945074Z"}
{"log":"\u0009\u003cautogenerated\u003e:1 +0x46 fp=0xc26aec5f00 sp=0xc26aec5ec0 pc=0x2292c66\n","stream":"stderr","time":"2024-10-08T13:10:41.095953971Z"}
{"log":"github.com/erigontech/erigon-lib/p2p/sentry.pumpStreamLoop[...].func2()\n","stream":"stderr","time":"2024-10-08T13:10:41.095977175Z"}
{"log":"\u0009github.com/erigontech/[email protected]/p2p/sentry/loop.go:126 +0x136 fp=0xc26aec5fe0 sp=0xc26aec5f00 pc=0x1fd4a96\n","stream":"stderr","time":"2024-10-08T13:10:41.095982765Z"}
{"log":"runtime.goexit({})\n","stream":"stderr","time":"2024-10-08T13:10:41.095997934Z"}
{"log":"\u0009runtime/asm_amd64.s:1700 +0x1 fp=0xc26aec5fe8 sp=0xc26aec5fe0 pc=0x49cf41\n","stream":"stderr","time":"2024-10-08T13:10:41.0960117Z"}
{"log":"created by github.com/erigontech/erigon-lib/p2p/sentry.pumpStreamLoop[...] in goroutine 296\n","stream":"stderr","time":"2024-10-08T13:10:41.096019434Z"}
{"log":"\u0009github.com/erigontech/[email protected]/p2p/sentry/loop.go:120 +0x287\n","stream":"stderr","time":"2024-10-08T13:10:41.096023923Z"}
{"log":"\n","stream":"stderr","time":"2024-10-08T13:10:41.096028181Z"}

Oct 09 '24 07:10 crebsy

hey, I just tried to re-sync from scratch with https://github.com/erigontech/erigon/releases/tag/v3.0.0-alpha5 and I'm still getting the same error. Is there any workaround to this so that the sync can complete ?

Oct 24 '24 08:10 crebsy

found a workaround for my case: remove --externalcl and use the internal caplin instead

Oct 26 '24 06:10 crebsy

any update here @domiwei @somnathb1? This is quite the bug to be fully reliant on Caplin when you're running existing systems with Lighthouse/etc

Nov 23 '24 10:11 86b

The fix worked for holesky, testing for mainnet

Nov 28 '24 22:11 AlexeyAkhunov

Looks like the fix is not complete, still getting MDBX_MAP_FULL: Environment mapsize limit reached on mainnet

Nov 29 '24 14:11 AlexeyAkhunov