reth icon indicating copy to clipboard operation
reth copied to clipboard

Published 20 hours ago • verified publisher icon paradigmxyz

Bus error

Open 0xDaizz opened this issue 1 year ago • 4 comments

Describe the bug

While I was syncing my node, unexpected error occured and stop my node continuously. It first happened yesterday, and my node still stops after that..

Steps to reproduce

run: RUST_LOG=info reth node --http --ws --metrics 127.0.0.1:9001

after some time, the node stops running

Node logs

$ RUST_LOG=info reth node --http --ws --metrics 127.0.0.1:9001
2023-06-21T05:54:02.033421Z  INFO reth::cli: reth 0.1.0 (a8be68a8) starting
2023-06-21T05:54:02.037196Z  INFO reth::cli: Configuration loaded path="/home/i7epr/.local/share/reth/mainnet/reth.toml"
2023-06-21T05:54:02.037248Z  INFO reth::cli: Opening database path="/home/i7epr/.local/share/reth/mainnet/db"
2023-06-21T05:54:02.046116Z  INFO reth::cli: Database opened
2023-06-21T05:54:02.046139Z  INFO reth::cli: Starting metrics endpoint addr=127.0.0.1:9001
2023-06-21T05:54:02.046963Z  INFO reth::cli: Transaction pool initialized
2023-06-21T05:54:02.047528Z  INFO reth::cli: Connecting to P2P network
2023-06-21T05:54:02.047820Z  INFO net::peers: Loading saved peers file=/home/i7epr/.local/share/reth/mainnet/known-peers.json
2023-06-21T05:54:02.050620Z  INFO reth::cli: Connected to P2P network peer_id=0x68d5…e859 local_addr=0.0.0.0:30303
2023-06-21T05:54:02.051814Z  INFO reth::cli: Consensus engine initialized
2023-06-21T05:54:02.052224Z  INFO reth::cli: Engine API handler initialized
2023-06-21T05:54:02.053160Z  INFO reth::cli: Status connected_peers=0 latest_block=0
2023-06-21T05:54:02.054168Z  INFO reth::cli: RPC auth server started
2023-06-21T05:54:02.054291Z  INFO reth::cli: RPC IPC server started url=/tmp/reth.ipc
2023-06-21T05:54:02.054312Z  INFO reth::cli: RPC HTTP server started url=0.0.0.0:8545
2023-06-21T05:54:02.054333Z  INFO reth::cli: RPC WS server started url=0.0.0.0:8546
2023-06-21T05:54:02.054353Z  INFO reth::cli: Starting consensus engine
2023-06-21T05:54:02.054683Z  INFO reth::node::events: Executing stage pipeline_stages=1/13 stage=Headers from=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.054683Z  INFO execute{stage=Headers}: sync::stages::headers: Target block already reached checkpoint=100.0% target=Hash(0x019ed8d11c72fbe5a7579ae9049fab59b585c0d476ce14b5bb88c914e6219c13)
2023-06-21T05:54:02.054798Z  INFO reth::node::events: Stage finished executing pipeline_stages=1/13 stage=Headers block=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055027Z  INFO reth::node::events: Executing stage pipeline_stages=2/13 stage=TotalDifficulty from=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055053Z  INFO reth::node::events: Stage finished executing pipeline_stages=2/13 stage=TotalDifficulty block=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055220Z  INFO reth::node::events: Executing stage pipeline_stages=3/13 stage=Bodies from=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055245Z  INFO reth::node::events: Stage finished executing pipeline_stages=3/13 stage=Bodies block=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055402Z  INFO reth::node::events: Executing stage pipeline_stages=4/13 stage=SenderRecovery from=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055424Z  INFO reth::node::events: Stage finished executing pipeline_stages=4/13 stage=SenderRecovery block=17501262 checkpoint=100.0% eta=unknown
2023-06-21T05:54:02.055577Z  INFO reth::node::events: Executing stage pipeline_stages=5/13 stage=Execution from=12583064 checkpoint=49.2% eta=unknown
2023-06-21T05:54:02.409138Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=1
2023-06-21T05:54:02.830241Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=1
2023-06-21T05:54:03.019037Z  INFO net: Session established remote_addr=157.90.35.166:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x4aeb4ab6c14b23e2c4cfdce879c04b0748a20d8e9b59e25ded2a08143e265c6c25936e74cbc8e641e3312ca288673d91f2f93f8e277de3cfa444ecdaaf982052 total_active=2
2023-06-21T05:54:32.053476Z  INFO reth::cli: Status connected_peers=1 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:54:47.290653Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=2
2023-06-21T05:54:47.619840Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=2
2023-06-21T05:55:02.052724Z  INFO reth::cli: Status connected_peers=1 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:55:23.654997Z  INFO net: Session established remote_addr=43.156.125.210:30304 client_version=erigon/v2.39.0-stable-b5891e2c/linux-amd64/go1.19.5 peer_id=0x482e6003b280a5fcc1f464fe677241098d111cd247e6e7a1f6aecbea03ae1129ca58489c1461c721814ea9550b2491e706d2f73fee8467f225c5b2ad7ed8d242 total_active=2
2023-06-21T05:55:24.248476Z  INFO net: Session established remote_addr=82.84.68.78:30404 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0x81d38f46b3dfe1a6243a7afac6ece3c2c0ed2e9393a3ee612c72f65de082518003fc42035e74aa3d28ec2119f81c33347db66f9ec3e85997520ba476281fa621 total_active=3
2023-06-21T05:55:24.354736Z  INFO net: Session established remote_addr=167.235.34.105:31383 client_version= peer_id=0xeeba78429de7f290e720925963b5e7dedc987dae2bcafcfc294b7fd214ab15588ebc4f80dff25508c784160d1c5fc3e86e8dd82e8bb33563b0957b1142290da5 total_active=4
2023-06-21T05:55:25.390370Z  INFO net: Session established remote_addr=131.153.171.234:30402 client_version= peer_id=0xa6e5e454ade25c2873cf654ae2e9a7e1ad6ed8d384cd8eb843d3f3ef7d9df47a00c2624c24a822325296cd18683e637fe4c29e05edd13bf5e1294d2a14057c41 total_active=5
2023-06-21T05:55:25.932743Z  INFO net: Session established remote_addr=13.112.74.179:30303 client_version=Geth/v1.11.6-stable/linux-amd64/go1.20.3 peer_id=0xb1f3d402608e063309b1473a23b59c57c8fedbaeb55b3cfadd5e065d40c7266524fee8e9dbf00ad6ba85d11083ea9a0a8be0d4155b52ebcd3b0df93c0cc93203 total_active=6
2023-06-21T05:55:26.179960Z  INFO net: Session established remote_addr=180.150.126.199:30303 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0xebf67584b0d7b7752afb621d56fd2e6bd012d764a51e1b116bb60a94ee1d17197f16f401d685c15d9e8f5520d636b64ae95cb63e20cd795531ed585de8963ae2 total_active=7
2023-06-21T05:55:26.242380Z  INFO net: Session established remote_addr=149.102.158.92:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x642a9136b2f9e9bec58651366ba3ed276dfb9e8353757c910a5ce27e77b396643b56f8949927059a5b9e4c8188f4447230a287f7cb528f06c07546c582426a8f total_active=8
2023-06-21T05:55:27.071257Z  INFO net: Session established remote_addr=46.4.28.27:30304 client_version=erigon/v2.43.0-stable-1106475e/linux-amd64/go1.20.3 peer_id=0x5aedd69a691490a422c785689a96492202a4d39c650b6963fc4ed50b81f2c369ded6b37029d9f7448d163fe210e188403a73078b1952e0755571d75420298521 total_active=9
2023-06-21T05:55:32.053098Z  INFO reth::cli: Status connected_peers=9 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:55:32.293396Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=10
2023-06-21T05:55:32.621317Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=10
2023-06-21T05:55:48.024420Z  INFO net: Session established remote_addr=82.84.68.78:30404 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0x81d38f46b3dfe1a6243a7afac6ece3c2c0ed2e9393a3ee612c72f65de082518003fc42035e74aa3d28ec2119f81c33347db66f9ec3e85997520ba476281fa621 total_active=9
2023-06-21T05:55:55.552826Z  INFO net: Session established remote_addr=167.235.34.105:31383 client_version= peer_id=0xeeba78429de7f290e720925963b5e7dedc987dae2bcafcfc294b7fd214ab15588ebc4f80dff25508c784160d1c5fc3e86e8dd82e8bb33563b0957b1142290da5 total_active=8
2023-06-21T05:55:56.473677Z  INFO net: Session established remote_addr=131.153.171.234:30402 client_version= peer_id=0xa6e5e454ade25c2873cf654ae2e9a7e1ad6ed8d384cd8eb843d3f3ef7d9df47a00c2624c24a822325296cd18683e637fe4c29e05edd13bf5e1294d2a14057c41 total_active=9
2023-06-21T05:56:02.053204Z  INFO reth::cli: Status connected_peers=9 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:56:03.234682Z  INFO net: Session established remote_addr=73.61.92.239:30404 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0x8ded8540f94a77e642166e089e3cf22b486c7dbea556f96a3fd32ad333f56012ac46b1ac8077ed0711705064aaa828100af6d0595e67ce8c96c1e6e2569d01aa total_active=10
2023-06-21T05:56:04.689373Z  INFO net: Session established remote_addr=176.9.31.184:30304 client_version=erigon/v2.42.0/linux-amd64/go1.19.7 peer_id=0x05ee87b945892c5b1baa7c62ef3e11c3b7b568f075e6f740e359691f27467d2160fae20ad700625f2ae18bbcefb2097ed001394f0876602a8bb24e6ba7c04d28 total_active=11
2023-06-21T05:56:04.755249Z  INFO net: Session established remote_addr=146.59.84.8:15701 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0xbbc7be9c85484b428d4742452523335ac60d15abd352cb50c69a465a9d3b003b1f59e6e08611fe7156826f841e112db56b21a3efe962cf3c7dcbae353ed7fea9 total_active=12
2023-06-21T05:56:06.305507Z  INFO net: Session established remote_addr=180.150.126.199:30303 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0xebf67584b0d7b7752afb621d56fd2e6bd012d764a51e1b116bb60a94ee1d17197f16f401d685c15d9e8f5520d636b64ae95cb63e20cd795531ed585de8963ae2 total_active=12
2023-06-21T05:56:06.793293Z  INFO net: Session established remote_addr=35.236.246.136:30303 client_version=Geth/v1.10.17-stable-25c9b49f/linux-amd64/go1.20 peer_id=0xa15981b6010c590d8d7f60ae027f7e698e77f8bcecde40d470ad13807b37917eb5cf90d07c291ca141f192325504c5ab4427f3f0280ceeed2ef5a215013be93a total_active=13
2023-06-21T05:56:17.300533Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=12
2023-06-21T05:56:18.185591Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=12
2023-06-21T05:56:18.366600Z  INFO net: Session established remote_addr=82.84.68.78:30404 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0x81d38f46b3dfe1a6243a7afac6ece3c2c0ed2e9393a3ee612c72f65de082518003fc42035e74aa3d28ec2119f81c33347db66f9ec3e85997520ba476281fa621 total_active=13
2023-06-21T05:56:27.345058Z  INFO net: Session established remote_addr=167.235.34.105:31383 client_version= peer_id=0xeeba78429de7f290e720925963b5e7dedc987dae2bcafcfc294b7fd214ab15588ebc4f80dff25508c784160d1c5fc3e86e8dd82e8bb33563b0957b1142290da5 total_active=11
2023-06-21T05:56:27.495194Z  INFO net: Session established remote_addr=131.153.171.234:30402 client_version= peer_id=0xa6e5e454ade25c2873cf654ae2e9a7e1ad6ed8d384cd8eb843d3f3ef7d9df47a00c2624c24a822325296cd18683e637fe4c29e05edd13bf5e1294d2a14057c41 total_active=12
2023-06-21T05:56:32.138970Z  INFO reth::cli: Status connected_peers=12 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:56:32.673896Z  INFO net: Session established remote_addr=94.45.205.162:30303 client_version=Nethermind/v1.17.3+671d60ad/linux-x64/dotnet7.0.4 peer_id=0xef80f896995048261669dd936904e749e16d89dd6e488c9a003b264d531ea568d78c1bef54f5d4491a890a9f7803d8a904a6430480500e1f7269673d25eef559 total_active=12
2023-06-21T05:56:49.587160Z  INFO net: Session established remote_addr=67.180.40.223:30303 client_version=Nethermind/v1.17.3+671d60ad/linux-x64/dotnet7.0.4 peer_id=0x271425f33e2f9bb5734be0e562e0bfcdae4038042e1226169e61079d8da4dd985655dc22191528d81be3baecd3157d2f6c36eafabe18fd27a2dd3ea8f2bb60c6 total_active=10
2023-06-21T05:56:58.519397Z  INFO net: Session established remote_addr=131.153.171.234:30402 client_version= peer_id=0xa6e5e454ade25c2873cf654ae2e9a7e1ad6ed8d384cd8eb843d3f3ef7d9df47a00c2624c24a822325296cd18683e637fe4c29e05edd13bf5e1294d2a14057c41 total_active=9
2023-06-21T05:56:58.713962Z  INFO net: Session established remote_addr=167.235.34.105:31383 client_version= peer_id=0xeeba78429de7f290e720925963b5e7dedc987dae2bcafcfc294b7fd214ab15588ebc4f80dff25508c784160d1c5fc3e86e8dd82e8bb33563b0957b1142290da5 total_active=10
2023-06-21T05:57:02.140228Z  INFO reth::cli: Status connected_peers=10 stage=Execution checkpoint=49.2% eta=unknown
2023-06-21T05:57:02.836762Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=11
2023-06-21T05:57:03.556021Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=10
2023-06-21T05:57:04.089806Z  INFO net: Session established remote_addr=94.45.205.162:30303 client_version=Nethermind/v1.17.3+671d60ad/linux-x64/dotnet7.0.4 peer_id=0xef80f896995048261669dd936904e749e16d89dd6e488c9a003b264d531ea568d78c1bef54f5d4491a890a9f7803d8a904a6430480500e1f7269673d25eef559 total_active=10
2023-06-21T05:57:19.026236Z  INFO net: Session established remote_addr=180.150.126.199:30303 client_version=Nethermind/v1.19.2+42ba08b7/linux-x64/dotnet7.0.5 peer_id=0xebf67584b0d7b7752afb621d56fd2e6bd012d764a51e1b116bb60a94ee1d17197f16f401d685c15d9e8f5520d636b64ae95cb63e20cd795531ed585de8963ae2 total_active=11
2023-06-21T05:57:29.525929Z  INFO net: Session established remote_addr=131.153.171.234:30402 client_version= peer_id=0xa6e5e454ade25c2873cf654ae2e9a7e1ad6ed8d384cd8eb843d3f3ef7d9df47a00c2624c24a822325296cd18683e637fe4c29e05edd13bf5e1294d2a14057c41 total_active=9
2023-06-21T05:57:29.812366Z  INFO net: Session established remote_addr=167.235.34.105:31383 client_version= peer_id=0xeeba78429de7f290e720925963b5e7dedc987dae2bcafcfc294b7fd214ab15588ebc4f80dff25508c784160d1c5fc3e86e8dd82e8bb33563b0957b1142290da5 total_active=10
2023-06-21T05:57:32.139993Z  INFO reth::cli: Status connected_peers=10 stage=Execution checkpoint=49.2% eta=unknown
Bus error

Platform(s)

Linux (x86)

What version/commit are you on?

$ reth --version reth Version: 0.1.0 Commit SHA: a8be68a8 Build Timestamp: 2023-06-17T18:08:42.213387486Z Build Features:

What database version are you on?

$ reth db version error: unrecognized subcommand 'version'

Usage: reth db [OPTIONS] <COMMAND>

For more information, try '--help'.

If you've built Reth from source, provide the full command you used

RUST_LOG=info reth node --http --ws --metrics 127.0.0.1:9001

Code of Conduct

  • [X] I agree to follow the Code of Conduct

0xDaizz avatar Jun 21 '23 06:06 0xDaizz

Do you have available disk space?

onbjerg avatar Jun 21 '23 09:06 onbjerg

Do you have available disk space?

yes, I still have 800GB more.

Currently I am running it on WSL. Will it be better if I re-install Reth on my windows via alpha version released yesterday?

0xDaizz avatar Jun 21 '23 13:06 0xDaizz

Please try with the alpha tag, just in case. When I last saw this error it was because I ran out of disk space on the partition that I had my DB on. Is it possible that you have more disk space, but on a different drive?

gakonst avatar Jun 21 '23 17:06 gakonst

Please try with the alpha tag, just in case. When I last saw this error it was because I ran out of disk space on the partition that I had my DB on. Is it possible that you have more disk space, but on a different drive?

Ok, I will try it with the alpha version, and update the results.

0xDaizz avatar Jun 22 '23 16:06 0xDaizz

@0xDaizz have you been able to repro this? closing optimistically, please re-open if still an issue & share more context if possible.

gakonst avatar Jul 03 '23 13:07 gakonst

@gakonst Tried on different computer, but encountered the same error, same stage(even similar percentage, Execution during 12.5 millionth block) Spec : Windows 11, Ryzen 5 5600G, 32GB RAM, Sumsung 970 Evo Plus 2Tb *2 RAID0 (total 4TB)

I ran the pre-built linux version (0.1.0 alpha.1), On my WSL2 Ubuntu 22.04.

$ RUST_LOG=info reth node --http --ws --metrics 127.0.0.1:9001
2023-07-05T12:07:43.982415Z  INFO reth::cli: reth 0.1.0-alpha.1 (b245193) starting
2023-07-05T12:07:43.984810Z  INFO reth::cli: Configuration loaded path="/home/daizz/.local/share/reth/mainnet/reth.toml"
2023-07-05T12:07:43.984831Z  INFO reth::cli: Opening database path="/home/daizz/.local/share/reth/mainnet/db"
2023-07-05T12:07:44.024962Z  INFO reth::cli: Database opened
2023-07-05T12:07:44.024987Z  INFO reth::cli: Starting metrics endpoint addr=127.0.0.1:9001
2023-07-05T12:07:44.030349Z  INFO reth::cli: Pre-merge hard forks (block based):
- Frontier                         @0
- Homestead                        @1150000
- Dao                              @1920000
- Tangerine                        @2463000
- SpuriousDragon                   @2675000
- Byzantium                        @4370000
- Constantinople                   @7280000
- Petersburg                       @7280000
- Istanbul                         @9069000
- MuirGlacier                      @9200000
- Berlin                           @12244000
- London                           @12965000
- ArrowGlacier                     @13773000
- GrayGlacier                      @15050000
Merge hard forks:
- Paris                            @58750000000000000000000 (network is not known to be merged)

Post-merge hard forks (timestamp based):
- Shanghai                         @1681338455

2023-07-05T12:07:44.033346Z  INFO reth::cli: Transaction pool initialized
2023-07-05T12:07:44.033677Z  INFO reth::cli: Connecting to P2P network
2023-07-05T12:07:44.039723Z  INFO reth::cli: Connected to P2P network peer_id=0xbdd1…7bfd local_addr=0.0.0.0:30303
2023-07-05T12:07:44.042125Z  INFO reth::cli: Consensus engine initialized
2023-07-05T12:07:44.042180Z  INFO reth::cli: Engine API handler initialized
2023-07-05T12:07:44.043087Z  INFO reth::cli: Status connected_peers=0 latest_block=0
2023-07-05T12:07:44.046098Z  INFO reth::cli: RPC auth server started
2023-07-05T12:07:44.046281Z  INFO reth::cli: RPC IPC server started url=/tmp/reth.ipc
2023-07-05T12:07:44.046301Z  INFO reth::cli: RPC HTTP server started url=0.0.0.0:8545
2023-07-05T12:07:44.046305Z  INFO reth::cli: RPC WS server started url=0.0.0.0:8546
2023-07-05T12:07:44.046320Z  INFO reth::cli: Starting consensus engine
2023-07-05T12:07:44.048261Z  INFO execute{stage=Headers}: sync::stages::headers: Target block already reached checkpoint=100.0% target=Hash(0x03e568c398a1d4cda19283cadb6b7f3d617ce50aab4328f1550eb8d552a7d17b)
2023-07-05T12:07:44.048269Z  INFO reth::node::events: Executing stage pipeline_stages=1/13 stage=Headers from=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048316Z  INFO reth::node::events: Stage finished executing pipeline_stages=1/13 stage=Headers block=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048520Z  INFO reth::node::events: Executing stage pipeline_stages=2/13 stage=TotalDifficulty from=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048534Z  INFO reth::node::events: Stage finished executing pipeline_stages=2/13 stage=TotalDifficulty block=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048667Z  INFO reth::node::events: Executing stage pipeline_stages=3/13 stage=Bodies from=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048682Z  INFO reth::node::events: Stage finished executing pipeline_stages=3/13 stage=Bodies block=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048824Z  INFO reth::node::events: Executing stage pipeline_stages=4/13 stage=SenderRecovery from=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048844Z  INFO reth::node::events: Stage finished executing pipeline_stages=4/13 stage=SenderRecovery block=17610872 checkpoint=100.0% eta=unknown
2023-07-05T12:07:44.048989Z  INFO reth::node::events: Executing stage pipeline_stages=5/13 stage=Execution from=12461508 checkpoint=47.5% eta=unknown
2023-07-05T12:07:44.354267Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=1
2023-07-05T12:07:44.813089Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=1
2023-07-05T12:07:45.511931Z  INFO net: Session established remote_addr=157.90.35.166:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x4aeb4ab6c14b23e2c4cfdce879c04b0748a20d8e9b59e25ded2a08143e265c6c25936e74cbc8e641e3312ca288673d91f2f93f8e277de3cfa444ecdaaf982052 total_active=1
2023-07-05T12:08:02.938293Z  INFO reth::node::events: Forkchoice updated head_block_hash=0x8888b2c9a547d5f4cfc2b38379ac0e2376d2e2ab00aff257f2daabce4aaea54b safe_block_hash=0x0060b62c7869a8b68ea3899d5cca2a860a2a7671bf1646a7e7797e769cd8bea7 finalized_block_hash=0x2250f870844dacf6b68872e131f363f932784f9c4b5e801080d7b863f11e5aef status=Syncing
2023-07-05T12:08:04.283399Z  INFO net: Session established remote_addr=3.113.221.160:30303 client_version=Geth/v1.11.5-stable-a38f4108/linux-amd64/go1.19 peer_id=0xcd64ec70e18b4f899127c4bd362dca5c977115f4986f69f75a8e7211e487857a7c7527d1fa4999a9f758d39473ba93bf6f5f3b63c622803c8028e315d91c9685 total_active=2
2023-07-05T12:08:14.043333Z  INFO reth::cli: Status connected_peers=1 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:08:29.277507Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=2
2023-07-05T12:08:29.607987Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=2
2023-07-05T12:08:44.042566Z  INFO reth::cli: Status connected_peers=1 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:08:50.987491Z  INFO net: Session established remote_addr=151.115.39.176:30303 client_version=Geth/v1.10.25-stable-69568c55/linux-amd64/go1.18.5 peer_id=0xc57eacb4455f84e8c1076cc8b24da60acb915759d8e50aafb2a461bc6617f3b7150edb38e45823539ededc219d1ee739dcf32f5a2da837258e4209850610f539 total_active=2
2023-07-05T12:08:51.969806Z  INFO net: Session established remote_addr=18.196.208.138:30303 client_version=Geth/v1.11.6-stable-ea9e62ca/linux-amd64/go1.20.3 peer_id=0x1e41de28ba40406a32aaac6c990ee535577fb4028bec5a723602bb65c59dcfcf1daf3b94325887ded034bb8fe96ee2e82cc1eb6d350dc06da2ada5078f2245ff total_active=2
2023-07-05T12:09:05.982845Z  INFO net: Session established remote_addr=15.204.162.53:30303 client_version=Geth/v1.11.5-stable/linux-amd64/go1.20.2 peer_id=0x971eaefe449b291f70570446b4c1b570adc2cf0f71ad067691a097970f249615fa09809133953160d2763ee3e6599494acdd54836d3fc013983e59c596c290f1 total_active=2
2023-07-05T12:09:06.487672Z  INFO net: Session established remote_addr=135.181.20.232:30303 client_version=erigon/v2.45.0/linux-amd64/go1.20 peer_id=0xabec809e941524326144e2e96487b4120c5c41d0f0fcd3a9c80b4b188687466ba9575cf2368d7dfc62adafb78e98f38fcd727380d17eddea7f005d968302a121 total_active=3
2023-07-05T12:09:06.912726Z  INFO net: Session established remote_addr=37.19.218.33:30000 client_version=Geth/v1.10.25-stable/linux-amd64/go1.17.10 peer_id=0xf95636b4fea568517e3f2939128ad1ae7c8066c250543a79f1e2ff21e6133ee3cdbac4d8984d46a0c0eb66ef861fefb099f5c3aa532635476a8d9df67bf71f2c total_active=4
2023-07-05T12:09:07.088735Z  INFO net: Session established remote_addr=5.161.195.102:30320 client_version= peer_id=0x487b3a99d860179da0dd861b361e487d9ffaa974a2a8dde0806dd3e23a4152c145a14b0d751b742edff512592fbf0f19372bc98242f4d13cac9e6d6a49689ce9 total_active=5
2023-07-05T12:09:07.292100Z  INFO net: Session established remote_addr=13.37.220.1:30325 client_version= peer_id=0x3710876f3e6b3102b6641345fea44e228c9802290349fa962523e8869e4e8b25b52fd94639b09bb33f5030ab5e617a81c7249de38bde5b4e3228f82c227453da total_active=6
2023-07-05T12:09:07.448967Z  INFO net: Session established remote_addr=35.236.246.136:30303 client_version=Geth/v1.10.17-stable-25c9b49f/linux-amd64/go1.20 peer_id=0xa15981b6010c590d8d7f60ae027f7e698e77f8bcecde40d470ad13807b37917eb5cf90d07c291ca141f192325504c5ab4427f3f0280ceeed2ef5a215013be93a total_active=7
2023-07-05T12:09:07.467185Z  INFO net: Session established remote_addr=58.243.201.60:30303 client_version=Geth/v1.11.1-stable-76961066/linux-amd64/go1.20.1 peer_id=0xc03608a07524d4fe5579b9ed53e131e8205a4d481c010fb27ab7db72b301a5901fb33ec0cfcaa8f8c6c1b2bfea6c5a4ac89d0730854a3f18746dce404cd71928 total_active=8
2023-07-05T12:09:07.637560Z  INFO net: Session established remote_addr=49.13.55.78:30303 client_version= peer_id=0x090855719b6f31993c3acfa941ffb589c6b8c57261848161de6019695c92a6b99a151ab3b09f1846db69716653c96d5d6fe41fbefbbc2724a989f902523b13de total_active=8
2023-07-05T12:09:08.550437Z  INFO net: Session established remote_addr=148.251.134.247:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.3 peer_id=0x454de14457d593cc6550def00e20a74591dee1d99d1dab877460a835e6a6c255e840df02645e812ef742f4492e051374d992bb2e7ca6e8370f5069738116c217 total_active=9
2023-07-05T12:09:14.042839Z  INFO reth::cli: Status connected_peers=9 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:09:14.600291Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=10
2023-07-05T12:09:29.892391Z  INFO net: Session established remote_addr=18.196.208.138:30303 client_version=Geth/v1.11.6-stable-ea9e62ca/linux-amd64/go1.20.3 peer_id=0x1e41de28ba40406a32aaac6c990ee535577fb4028bec5a723602bb65c59dcfcf1daf3b94325887ded034bb8fe96ee2e82cc1eb6d350dc06da2ada5078f2245ff total_active=10
2023-07-05T12:09:44.043137Z  INFO reth::cli: Status connected_peers=6 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:09:59.272932Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=7
2023-07-05T12:09:59.624796Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=7
2023-07-05T12:10:14.043002Z  INFO reth::cli: Status connected_peers=5 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:10:25.049023Z  INFO net: Session established remote_addr=3.130.163.224:30324 client_version= peer_id=0x4b65d41f2591cd83768995e9e53b516506f317db8da210375b299860a39a7dd412fa214ecfff01f3e6ac90eb5ec3ff493823a46aed0df5cc9b8cafd4d92ec8db total_active=6
2023-07-05T12:10:25.734523Z  INFO net: Session established remote_addr=18.234.197.203:30303 client_version= peer_id=0xd1f604d01f2a35ad620e4852893b787d0123c9f7706146db17052d560903828a91e0d539e496e9ab51aa173944e7d709dedf4a610a65067e749781643236b291 total_active=7
2023-07-05T12:10:27.716649Z  INFO net: Session established remote_addr=35.158.189.165:30316 client_version= peer_id=0xbb96d22b99ad8e56e05317b06923ca2ebc73e54f8f859637cf4fc0cbf0174fa7c4851684f7b79344032e02f84db029ca64b43ee35d8a9d856d5881c7a708c197 total_active=8
2023-07-05T12:10:27.937886Z  INFO net: Session established remote_addr=49.13.55.78:30323 client_version= peer_id=0x857fc46c379a8f2ceb8316bb5c9d14d7ad40633c6b975e8fb826e15e05a8b11958bbd9152bb88df11b404487f41910efc2f7593a46d9bf125a005c540d95ddbf total_active=9
2023-07-05T12:10:29.574045Z  INFO net: Session established remote_addr=119.236.146.20:30404 client_version=Nethermind/v1.19.3+e8ac1da4/linux-x64/dotnet7.0.8 peer_id=0x1a6a6decd9b17b714e583ae203b5b1f2c1aa3c956c252c097beb93ff792ea55036c1388ac57d46c5c2a58bbfda3fc4dc87c03eae45766a97ef13565daffc9643 total_active=10
2023-07-05T12:10:44.042491Z  INFO reth::cli: Status connected_peers=10 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:10:44.283254Z  INFO net: Session established remote_addr=18.138.108.67:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0xd860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666 total_active=11
2023-07-05T12:10:44.927024Z  INFO net: Session established remote_addr=135.181.20.232:30303 client_version=erigon/v2.45.0/linux-amd64/go1.20 peer_id=0xabec809e941524326144e2e96487b4120c5c41d0f0fcd3a9c80b4b188687466ba9575cf2368d7dfc62adafb78e98f38fcd727380d17eddea7f005d968302a121 total_active=11
2023-07-05T12:10:45.665026Z  INFO net: Session established remote_addr=3.209.45.79:30303 client_version=Geth/v1.12.0-stable-e501b3b0/linux-amd64/go1.20.4 peer_id=0x22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de total_active=12
2023-07-05T12:11:00.521313Z  INFO net: Session established remote_addr=18.196.208.138:30303 client_version=Geth/v1.11.6-stable-ea9e62ca/linux-amd64/go1.20.3 peer_id=0x1e41de28ba40406a32aaac6c990ee535577fb4028bec5a723602bb65c59dcfcf1daf3b94325887ded034bb8fe96ee2e82cc1eb6d350dc06da2ada5078f2245ff total_active=7
2023-07-05T12:11:14.042459Z  INFO reth::cli: Status connected_peers=6 stage=Execution checkpoint=47.5% eta=unknown
2023-07-05T12:11:16.228493Z  INFO net: Session established remote_addr=47.95.116.123:30303 client_version=Geth/v1.12.0-stable-6d7960aa/linux-amd64/go1.20.3 peer_id=0x36f0a7c11c5c445933b73870b16423c954bed5543da2170365f169570c77b1ffafbf886574b764d04a3f8bb0a61b298cb7261efb6223fefa62bc4887599a628b total_active=7
Bus error

0xDaizz avatar Jul 05 '23 12:07 0xDaizz

It seems that WSL2 might have some issues with the virtual memory stuff that MDBX does. I am not entirely sure how to debug this - essentially, a bus error (SIGBUS) is raised by hardware when a process tries to access memory that the CPU cannot physically address (an invalid address for the address bus)

It can also be caused by a hardware error but since you tried on a second computer this should be unlikely. We'd need to figure out how virtual memory works in WSL2

onbjerg avatar Jul 05 '23 12:07 onbjerg

@0xDaizz Can you retry this on alpha.3? Is this still an issue? Please also confirm your WSL version and exact command.

gakonst avatar Jul 10 '23 14:07 gakonst

@0xDaizz Can you retry this on alpha.3? Is this still an issue? Please also confirm your WSL version and exact command.

I always use WSL version 2. I cannot use windows for a while(cuz I removed windows to install ubuntu), so maybe I can try on windows or WSL some time later.

I feel I should upgrade my desktop hardwares so gonna make up new computer soon. Maybe I can test then!

I will follow up this issue cuz I am more familiar with Windows than linux. but the linux/MAC Cli is much better than Powershell so I want to run reth on WSL2..

anyway, thanks for helping. will be back with this issue if there are any problems!

0xDaizz avatar Jul 11 '23 19:07 0xDaizz

Roger - closing for now, please re-open if it arises again.

gakonst avatar Jul 11 '23 20:07 gakonst

@gakonst

here comes the error again.

With another windows 10 computer, WSL 2.

I think there is some problem with this range(45~50%, block of 12500000 of execution phase.

2023-07-23T07:07:38.119665Z  INFO reth::cli: Status connected_peers=90 stage=Execution checkpoint=45.7% eta=2days 48m 53s
2023-07-23T07:07:43.892160Z  INFO reth::node::events: Forkchoice updated head_block_hash=0xc2ffc8b0840c8aa0deaa9617fae79686eb6710197a7762175fbaa648591b3028 safe_block_hash=0x2ec5b9658a274d609a3e9fce4e5e683bf9b94a70287aff62c74c814ca54590c0 finalized_block_hash=0xbaaacd2b6f7b9ef070a15900d368c9d182a828b065318fd5834cbf7542d0515e status=Syncing
2023-07-23T07:07:55.243550Z  INFO reth::node::events: Forkchoice updated head_block_hash=0x198f2d23447ea4da528802007fa07d209d922a209ece2449f37b8ff90789b59f safe_block_hash=0x2ec5b9658a274d609a3e9fce4e5e683bf9b94a70287aff62c74c814ca54590c0 finalized_block_hash=0xbaaacd2b6f7b9ef070a15900d368c9d182a828b065318fd5834cbf7542d0515e status=Syncing
2023-07-23T07:08:06.862581Z  INFO reth::node::events: Forkchoice updated head_block_hash=0xaf2907d629246758028cbc1ce158faf3a2726904c9af6004e92872a3487c21b4 safe_block_hash=0x2ec5b9658a274d609a3e9fce4e5e683bf9b94a70287aff62c74c814ca54590c0 finalized_block_hash=0xbaaacd2b6f7b9ef070a15900d368c9d182a828b065318fd5834cbf7542d0515e status=Syncing
2023-07-23T07:08:08.119129Z  INFO reth::cli: Status connected_peers=90 stage=Execution checkpoint=45.7% eta=2days 48m 23s
Bus error

0xDaizz avatar Jul 23 '23 07:07 0xDaizz

Fyi:

PC spec: Ryzen 7900X, 32GB RAM, WD SN850X 4TB

Reth build: 0.1.0-alpha.4

Build command : RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf

Node execution command: RUST_LOG=info ~/reth/target/maxperf/reth node --datadir ~/data/reth_data --authrpc.jwtsecret ~/data/jwt.hex --ws --ws.addr="127.0.0.1" --ws.api=eth,web3,net,txpool --http --http.api=eth,web3,net,txpool --http.addr="127.0.0.1" --http.port=8545 --metrics 127.0.0.1:9001

0xDaizz avatar Jul 23 '23 07:07 0xDaizz

@gakonst @onbjerg I managed to find what was wrong: The default WSL2 virtual disk space was set to 1TB maximum.

So I followed this instruction and the bus error problem got solved!

https://learn.microsoft.com/en-us/windows/wsl/disk-space#how-to-expand-the-size-of-your-wsl-2-virtual-hard-disk

I suggest that you add this information on the Reth book! thanks!!

0xDaizz avatar Sep 02 '23 23:09 0xDaizz