nimbus-eth2 icon indicating copy to clipboard operation
nimbus-eth2 copied to clipboard

Buffer incoming attestations/syncmsgs/etc for which the block has still not arrive

Open arnetheduck opened this issue 1 year ago • 1 comments

Because of network delays as async validation, attestations and other small messages may arrive earlier than what we anticipate for the block - we should buffer such attestations and validate them on block arrival instead of IGNORE:ing them as we do now.

This problem has become worse since adding the execution client into the mix which removes the "implicit" queue nimbus had due to its synchronous processing, since EL verification now happens asynchronously.

With the execution client extending the time it takes to validate a block, this has increasingly been a source of "lost" attestations.

Per-hour for example, this node ignored 7053 attestations:

$ grep "Attestation block unknown" nbc_bn_20230616050000.txt -c
7053

Some may have been for truly unknown blocks but this is rare per manual verification.

The number of attestations we need to buffer is fairly well-bounded by the spec - validator_count / 32 and the buffer can be discarded every slot.

arnetheduck avatar Jun 16 '23 06:06 arnetheduck

Whatever is done here should ensure it doesn't reintroduce https://github.com/status-im/nimbus-eth2/issues/1582

tersec avatar Jun 16 '23 14:06 tersec