besu
besu copied to clipboard
Sync block body
PR description
This PR changes the SNAP and CHECKPOINT sync to reduce the memory requirements by not decoding the block bodies requested from peers. This results in less memory and CPU usage during the sync. This PR does make sure that the properties of the syncs are the same as before, e.g. transaction indexing and block added events haven't changed.
Note: this PR does not significantly reduce the sync time on it's own, because the bottleneck is the import step of the download pipeline. It will however allow us to increase the parallelism of the pipeline because of the reduced memory and CPU usage. Once the import step is improved that increase in parallelism will allow us to reduce the sync time.
Note: Not decoding receipts during the sync does improve the memory and CPU usage as well. This has been shown in a POC.
First screenshot:
CPU% GC young generation during sync with this PR:
CPU% GC young generation during sync without this PR:
Metrics during the sync: https://grafana.o11y.web3factory.consensys.net/d/_Mqt4ksnz/besu-full?from=2025-02-04T05:46:27.668Z&orgId=19&refresh=5s&timezone=utc&to=2025-02-05T19:09:47.777Z&var-Filters=&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B&var-executor=$__all&var-overview_engine_percentiles=$__all&var-protocol_inbound=$__all&var-protocol_outbound=$__all&var-sync_task_quantile=$__all&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0005&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0004&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0003&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0002&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-control-0001&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0005&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0004&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0003&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0002&var-system=dev-elc-bu-nb-mainnet-stefan-sync-withSBB-0001&var-txpool_layer=$__all&var-vertx_pool_name=$__all
This pr is stale because it has been open for 30 days with no activity.
This pr was closed because it has been inactive for 14 days since being marked as stale.
This pr is stale because it has been open for 30 days with no activity.