reth icon indicating copy to clipboard operation
reth copied to clipboard

OOM during sync

Open punk5736 opened this issue 2 years ago • 0 comments

Describe the bug

There is likely a memory leak in the reth syncing pipeline. With 128GB RAM available, the process has failed due to OOM errors.

I did not get the commit hash which produced the OOM, but after pulling main 9e72cbf6b44fd195738c6d85f461304578c8156c, recompiling, and restarting the process has already ballooned to ~90GB.

Screen Shot 2023-06-22 at 12 45 53 PM

Steps to reproduce

$ git checkout 9e72cbf6b44fd195738c6d85f461304578c8156c

$ RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf

$ reth node

Node logs

Here is the latest log with a block referenced in case the sync point helps is useful.

Jun 22 17:47:38 reth[149860]: 2023-06-22T17:47:38.412442Z  INFO try_insert_validated_block{block=(17536667, 0xf73ffd11d424742f9d0b303cc461db8df69c38772e1c43a82feff5b25f0ae691)}:try_append_canonical_chain:try_insert_validated_block{block=(17536688, 0xa67e40d64f49c893bff382d1a4d910c6a9a8c8c0ace13ebea4782e6c05e3efa5)}: blockchain_tree: return=Ok(Valid)

Platform(s)

Linux (x86)

What version/commit are you on?

9e72cbf6b44fd195738c6d85f461304578c8156c

What database version are you on?

Current database version: 1 Local database version: 1

If you've built Reth from source, provide the full command you used

RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf

Code of Conduct

  • [X] I agree to follow the Code of Conduct

punk5736 avatar Jun 22 '23 17:06 punk5736