[fusaka-holesky] Proposer lookahead processing times increasing epoch transition

[philknows]

Describe the bug

Proposer lookahead processing takes quite a bit of time - upwards of ~300-350ms to the transition processing.

Luca from Serenita reports seeing similar values and block proposing is affected by seeing proposed blocks come later in the slot.

Expected behavior

Proposer lookahead should have lowest overhead possible to minimize epoch transition processing.

Steps to reproduce

No response

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

v1.35.0-rc.1

[twoeths]

we always compute shuffling synchronously and don't use the async build in beforeEpochTransition postfulu

this shuffling calculation time by source confirm that

prefulu, we have an assumption that next epoch shuffling is not needed right away and can be lazily computed postfulu, that assumption does not work since we add proposerLookahead to BeaconState, hence we need the next shuffling computation right away

I don't see how to save epoch transition time without turning epoch transition functions to async

[twoeths]

attaching the profile persisted in the above image

holesky_fulu_oct_02.cpuprofile.zip