perf(consensus): never drop FCU

[emhane]

Closes https://github.com/paradigmxyz/reth/issues/8203.

• Adds VecDeque with engine messages to ensure no messages a dropped.

[emhane]

wondering if it's tackling the problem at head better, to replace this with a VecDeque instead @mattsse

https://github.com/paradigmxyz/reth/blob/081796b138fd00a6d0af2ad463123fdbe04765ab/crates/consensus/beacon/src/engine/mod.rs#L187-L188

[mattsse]

wondering if it's tackling the problem at head better, to replace this with a VecDeque instead @mattsse

https://github.com/paradigmxyz/reth/blob/081796b138fd00a6d0af2ad463123fdbe04765ab/crates/consensus/beacon/src/engine/mod.rs#L187-L188

eventually, yes, but rn this is basically just fake concurrency of sequential message processing as prep for #7154 but maybe a vecque could also work here @rkrasiuk

[shekhirin]

That looks good and should work.

I was only thinking maybe we should only add the messages to the queue when we actually can't process them, i.e. when the pruner is running and we skip an FCU. Instead of always caching the messages and popping them from the queue on the next poll that I think slows down the engine message processing a bit.

[emhane]

doesn't work though :/ @shekhirin @mattsse and not sure why, that loop is very complex

[emhane]

think we should rewrite this future implementation to be a stream, so we can better reason about it. rn it's very complex to create a mental model of how this state machine will behave based on varying external conditions. related https://github.com/paradigmxyz/reth/issues/6541.

[emhane]

closing in favour of https://github.com/paradigmxyz/reth/pull/8315