nimbus-eth2
nimbus-eth2 copied to clipboard
Buffer incoming attestations/syncmsgs/etc for which the block has still not arrive
Because of network delays as async validation, attestations and other small messages may arrive earlier than what we anticipate for the block - we should buffer such attestations and validate them on block arrival instead of IGNORE:ing them as we do now.
This problem has become worse since adding the execution client into the mix which removes the "implicit" queue nimbus had due to its synchronous processing, since EL verification now happens asynchronously.
With the execution client extending the time it takes to validate a block, this has increasingly been a source of "lost" attestations.
Per-hour for example, this node ignored 7053 attestations:
$ grep "Attestation block unknown" nbc_bn_20230616050000.txt -c
7053
Some may have been for truly unknown blocks but this is rare per manual verification.
The number of attestations we need to buffer is fairly well-bounded by the spec - validator_count / 32 and the buffer can be discarded every slot.
Whatever is done here should ensure it doesn't reintroduce https://github.com/status-im/nimbus-eth2/issues/1582