besu icon indicating copy to clipboard operation
besu copied to clipboard

Sync block body

Open pinges opened this issue 10 months ago • 3 comments

PR description

This PR changes the SNAP and CHECKPOINT sync to reduce the memory requirements by not decoding the block bodies requested from peers. This results in less memory and CPU usage during the sync. This PR does make sure that the properties of the syncs are the same as before, e.g. transaction indexing and block added events haven't changed.

Note: this PR does not significantly reduce the sync time on it's own, because the bottleneck is the import step of the download pipeline. It will however allow us to increase the parallelism of the pipeline because of the reduced memory and CPU usage. Once the import step is improved that increase in parallelism will allow us to reduce the sync time.

Note: Not decoding receipts during the sync does improve the memory and CPU usage as well. This has been shown in a POC.

First screenshot: ScShotWithSBB

CPU% GC young generation during sync with this PR: withSBB1_GCJ

CPU% GC young generation during sync without this PR: WithoutSBB-GCJ-04

Metrics during the sync: https://grafana.o11y.web3factory.consensys.net/d/_Mqt4ksnz/besu-full?from=2025-02-04T05:46:27.668Z&orgId=19&refresh=5s&timezone=utc&to=2025-02-05T19:09:47.777Z&var-Filters=&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B&var-executor=$__all&var-overview_engine_percentiles=$__all&var-protocol_inbound=$__all&var-protocol_outbound=$__all&var-sync_task_quantile=$__all&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0005&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0004&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0003&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0002&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0001&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0005&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0004&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0003&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0002&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0001&var-txpool_layer=$__all&var-vertx_pool_name=$__all

pinges avatar Feb 04 '25 06:02 pinges

This pr is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Mar 30 '25 02:03 github-actions[bot]

This pr was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Apr 13 '25 02:04 github-actions[bot]

This pr is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Jun 19 '25 02:06 github-actions[bot]