feat: calculate state root in background

[keefel]

trafficstars

Description

This PR adds an option to calculate the state root in the background, improving block insertion performance during live sync.

Rationale

Calculating the state root during live sync takes a lot of time, resulting in poor performance. This feature allows for background calculation of the state root and merges calculations for multiple blocks to update the trie simultaneously. Our tests show that enabling this feature can improve performance from 50 mgasps to 400 mgasps.

Without this feature:

With this feature enabled:

Example

NA

Changes

Notable changes:

• Calculate the state root in the background.
• Merge the calculation of the state root for multiple blocks.

Potential Impacts

• The persistent database should align with the background calculation of the state root.
• When unwinding blocks, you can only unwind to the block where the state root has been calculated.

[keefel]

thanks for this,

the way this is implemented isn't something we want to include as is.

the node's main loop can already be customized, although we definitely need to make this more convenient.

and it'd be nice to reuse the existing engine/tree logic but make it customizable. so what I'd like to do instead is narrow down what needs to be configurable. it looks like you want a modified on_new_payload function essentially?

so what I'd try instead is making this part configurable via some trait based (trait Handler) approach, basically a type that's responsible for this

before you spend more time on these features, I'd recommend opening an issue for this feature so we can come up with an appropriate solution for this

Thank you for your response. I will open an issue for this feature.