nethermind
nethermind copied to clipboard
NethermindEth
Node stuck with sync when PivotUpdator changed block for second time based on FCU
At the beginning of the sync for long time there were messages like this:
Pivot block from Consensus Layer too far from head. PivotBlockNumber: 7087258, TargetBlockNumber: 7087633, difference: 375 blocks. Max difference allowed: 128
After that, I've noticed that new pivot block was found
23 Nov 11:34:20 | New pivot block has been set based on ForkChoiceUpdate from CL. Pivot block number: 7087603, hash: 0x473ca9b477646669691303e0d0649c0c2936872b50f4c75b3f84311720ae35ca
23 Nov 11:34:21 | Changing state UpdatingPivot to FastHeaders at processed: 0 | state: 0 | block: 0 | header: 0 | target block: 7087633 | peer block: 7087633
From the moment of finding new block I was getting those errors:
23 Nov 11:34:22 | Unable to find beacon header at height 7087270. This is unexpected, forcing a new beacon sync.
It seems like PivotBlockNumber is still set to 7087270 where new PivotBlockNumber after some time was set to: 7087603
More context: At the very beginning, node is expected to be in mode UpdatingPivot. Somehow node is in both UpdatingPivot and BeaconHeaders - this way it set Beacon Pivot before successful update of general pivot (via PivotUpdator). In most of the cases it doesn't matter, but it is breaking sync if there is an initial lag on CL side. We need to figure out why nethermind is enabling BeaconHeaders before the end of UpdatingPivot. It doesn't look possible in MultiSyncModeSelector, looks like there is some backdoor way of triggering it
hi there,
right now i'm observing such issue - just tried to sync from the scratch with newest bins ("nimbus" for CL)
Mar 15 20:14:59 eth-cyfradc-01 nethermind[1117679]: 2024-03-15 20:14:59.8635|INFO|Pivot block from Consensus Layer too far from head. PivotBlockNumber: 19433300, TargetBlockNumber: 19442062, difference: 8762 blocks. Max difference allowed: 128
