Execution Payload is never included if TTD is reached before Bellatrix

[marioevz]

Describe the bug In hive a setup, it's possible to configure the total difficulty to a value that will be reached before the Bellatrix epoch is reached by the beacon chain.

When this happens, the blocks produced never include an Execution Payload and therefore the execution chain halts.

TTD (300) block mined by geth:

INFO [07-06|22:23:26.001] 🔨 mined potential block                  number=150 hash=93ade3..e05338

Bellatrix Slot 64 (Epoch 2), 2 minutes later:

INF 2022-07-06 22:25:19.000+00:00 Slot start                                 topics="beacnde" slot=64 epoch=2 sync=synced peers=2 head=bc76ab11:63 finalized=0:00000000 delay=379us944ns

Afterwards, slot 134+ still has zero exec payload:

beacon 0: slot=134, head=0x8b60..5279, health=0.17, exec_payload=0x0000..0000
beacon 1: slot=134, head=0x8b60..5279, health=0.17, exec_payload=0x0000..0000
beacon 2: slot=134, head=0x8b60..5279, health=0.17, exec_payload=0x0000..0000

If Bellatrix happens before the TTD, the logs show:

INF 2022-07-06 22:47:43.598+00:00 startEth1Syncing: checking for merge terminal block topics="eth1" currentEpoch=2 BELLATRIX_FORK_EPOCH=2 totalDifficulty=416 ttd=500 terminalBlockHash=None[BlockHash]

However, if TTD is reached before Bellatrix, logs never show startEth1Syncing.

To Reproduce Using branch https://github.com/marioevz/hive/tree/all-tests-plus-cl-clients, run:

./hive --client go-ethereum,nimbus-bn,nimbus-vc --sim eth2/engine --sim.loglevel 5

Test ttd-before-bellatrix-nimbus-go-ethereum will fail.

Additional context

• Hive setup uses HTTP instead of WS
• eth.getBlockByNumber("latest") to the execution client returns the last PoW block, so the client is not syncing
• Hive configuration script can be found here: https://github.com/marioevz/hive/blob/all-tests-plus-cl-clients/clients/nimbus-bn/nimbus_bn.sh

Nimbus version: Nimbus/v22.6.1-e18305-stateofus