erigon icon indicating copy to clipboard operation
erigon copied to clipboard

Published 20 hours ago • verified publisher icon erigontech

Polygon Full Sync Crashing

Open quickchase opened this issue 2 years ago • 3 comments

System information

Erigon version: erigon version 2022.05.5-alpha-ff5b94ee

OS & Version: Linux

Commit hash : ff5b94ee

Expected behaviour

Don't crash

Actual behaviour

Crash

Steps to reproduce the behaviour

WorkingDirectory=/home/matic
ExecStart=/usr/bin/erigon --syncmode=full --chain=bor-mainnet --db.pagesize 64kb --datadir=/home/matic/datadir --ethash.dagdir=/home/matic/ethash --bor.heimdall="http://localhost:1317" --http --http.addr="0.0.0.0" --http.compression --http.vhosts="*" --http.corsdomain="*" --http.api=eth,debug,net,trace,web3,erigon,bor --ws --ws.compression --rpc.gascap=300000000

Backtrace

This output is trimmed as it repeats over and over.

goroutine 78334 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78812 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78413 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78821 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78923 [select]:
 net.(*netFD).connect.func2()
         net/fd_unix.go:118 +0x86
 created by net.(*netFD).connect
         net/fd_unix.go:117 +0x37b
 goroutine 78633 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78513 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78478 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78762 [select]:
 net.(*netFD).connect.func2()
         net/fd_unix.go:118 +0x86
 created by net.(*netFD).connect
         net/fd_unix.go:117 +0x37b
 goroutine 78552 [select]:
 github.com/ledgerwatch/erigon/p2p.(*Peer).run(0xc014184460)
         github.com/ledgerwatch/erigon/p2p/peer.go:249 +0x23e
 github.com/ledgerwatch/erigon/p2p.(*Server).runPeer(0xc000388840, 0xc014184460)
         github.com/ledgerwatch/erigon/p2p/server.go:1074 +0x1e5
 created by github.com/ledgerwatch/erigon/p2p.(*Server).launchPeer
         github.com/ledgerwatch/erigon/p2p/server.go:1056 +0x118
 goroutine 78564 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78574 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78602 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78873 [IO wait]:
 internal/poll.runtime_pollWait(0x710e47a6c788, 0x77)
         runtime/netpoll.go:302 +0x89
 internal/poll.(*pollDesc).wait(0xc033ac5b80?, 0x0?, 0x0)
         internal/poll/fd_poll_runtime.go:83 +0x32
 internal/poll.(*pollDesc).waitWrite(...)
         internal/poll/fd_poll_runtime.go:92
 internal/poll.(*FD).WaitWrite(...)
         internal/poll/fd_unix.go:741
 net.(*netFD).connect(0xc033ac5b80, {0x1f8e488?, 0xc002d70ea0}, {0xc0368c78e0?, 0x43e0a5?}, {0x1f802a0?, 0xc0332b7300?})
         net/fd_unix.go:141 +0x716
 net.(*netFD).dial(0xc033ac5b80, {0x1f8e488, 0xc002d70ea0}, {0x1f93a00?, 0x0?}, {0x1f93a00?, 0xc07ef8d200}, 0x0?)
         net/sock_posix.go:149 +0x394
 net.socket({0x1f8e488, 0xc002d70ea0}, {0x195826d, 0x3}, 0x2, 0x1, 0x0?, 0xd0?, {0x1f93a00, 0x0}, ...)
         net/sock_posix.go:70 +0x2b2
 net.internetSocket({0x1f8e488, 0xc002d70ea0}, {0x195826d, 0x3}, {0x1f93a00, 0x0}, {0x1f93a00, 0xc07ef8d200}, 0xc1a59057a0?, 0x0, ...)
         net/ipsock_posix.go:142 +0xf8
 net.(*sysDialer).doDialTCP(0xc033ac5b00, {0x1f8e488, 0xc002d70ea0}, 0x0, 0x17281a0?)
         net/tcpsock_posix.go:65 +0xa5
 net.(*sysDialer).dialTCP(0x2d11c21?, {0x1f8e488?, 0xc002d70ea0?}, 0xc0368c7b70?, 0x49df3d?)
         net/tcpsock_posix.go:61 +0x59
 net.(*sysDialer).dialSingle(0xc033ac5b00, {0x1f8e488, 0xc002d70ea0}, {0x1f86680?, 0xc07ef8d200})
         net/dial.go:583 +0x28b
 net.(*sysDialer).dialSerial(0xc033ac5b00, {0x1f8e488, 0xc002d70ea0}, {0xc028f7ac50?, 0x1, 0x1958b57?})
         net/dial.go:551 +0x312
 net.(*Dialer).DialContext(0xc002846480, {0x1f8e418, 0xc0000b5840}, {0x195826d, 0x3}, {0xc02e280f90, 0x13})
         net/dial.go:428 +0x736
 github.com/ledgerwatch/erigon/p2p.tcpDialer.Dial({0x164de60?}, {0x1f8e418, 0xc0000b5840}, 0x164de60?)
         github.com/ledgerwatch/erigon/p2p/dial.go:69 +0x5a
 github.com/ledgerwatch/erigon/p2p.(*dialTask).dial(0xc07ef8d0b0, 0xc002608580, 0xc0eb872e10)
         github.com/ledgerwatch/erigon/p2p/dial.go:545 +0x67
 github.com/ledgerwatch/erigon/p2p.(*dialTask).run(0xc07ef8d0b0, 0x0?)
         github.com/ledgerwatch/erigon/p2p/dial.go:495 +0x6b
 github.com/ledgerwatch/erigon/p2p.(*dialScheduler).startDial.func1()
         github.com/ledgerwatch/erigon/p2p/dial.go:466 +0x51
 created by github.com/ledgerwatch/erigon/p2p.(*dialScheduler).startDial
         github.com/ledgerwatch/erigon/p2p/dial.go:464 +0x3f2
 goroutine 78654 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78417 [IO wait]:
 internal/poll.runtime_pollWait(0x7f0f0c33ce18, 0x77)
         runtime/netpoll.go:302 +0x89
 internal/poll.(*pollDesc).wait(0xc00321a880?, 0x0?, 0x0)
         internal/poll/fd_poll_runtime.go:83 +0x32
 internal/poll.(*pollDesc).waitWrite(...)
         internal/poll/fd_poll_runtime.go:92
 internal/poll.(*FD).WaitWrite(...)
         internal/poll/fd_unix.go:741
 net.(*netFD).connect(0xc00321a880, {0x1f8e488?, 0xc0a0cdf5c0}, {0xc0186e18e0?, 0x43e0a5?}, {0x1f802a0?, 0xc0156efa20?})
         net/fd_unix.go:141 +0x716
 net.(*netFD).dial(0xc00321a880, {0x1f8e488, 0xc0a0cdf5c0}, {0x1f93a00?, 0x0?}, {0x1f93a00?, 0xc01a08e5a0}, 0x0?)
         net/sock_posix.go:149 +0x394
 net.socket({0x1f8e488, 0xc0a0cdf5c0}, {0x195826d, 0x3}, 0x2, 0x1, 0x0?, 0xd0?, {0x1f93a00, 0x0}, ...)
         net/sock_posix.go:70 +0x2b2
 net.internetSocket({0x1f8e488, 0xc0a0cdf5c0}, {0x195826d, 0x3}, {0x1f93a00, 0x0}, {0x1f93a00, 0xc01a08e5a0}, 0xc11e9a1160?, 0x0, ...)
         net/ipsock_posix.go:142 +0xf8
 net.(*sysDialer).doDialTCP(0xc00321a800, {0x1f8e488, 0xc0a0cdf5c0}, 0x0, 0x17281a0?)
         net/tcpsock_posix.go:65 +0xa5
 net.(*sysDialer).dialTCP(0x2d11c21?, {0x1f8e488?, 0xc0a0cdf5c0?}, 0xc0186e1b70?, 0x49df3d?)
         net/tcpsock_posix.go:61 +0x59
 net.(*sysDialer).dialSingle(0xc00321a800, {0x1f8e488, 0xc0a0cdf5c0}, {0x1f86680?, 0xc01a08e5a0})
         net/dial.go:583 +0x28b
 net.(*sysDialer).dialSerial(0xc00321a800, {0x1f8e488, 0xc0a0cdf5c0}, {0xc00712c300?, 0x1, 0x1958b57?})
         net/dial.go:551 +0x312
 net.(*Dialer).DialContext(0xc002846480, {0x1f8e418, 0xc0000b5840}, {0x195826d, 0x3}, {0xc0cb5e6d98, 0x11})
         net/dial.go:428 +0x736
 github.com/ledgerwatch/erigon/p2p.tcpDialer.Dial({0x164de60?}, {0x1f8e418, 0xc0000b5840}, 0x164de60?)
         github.com/ledgerwatch/erigon/p2p/dial.go:69 +0x5a
 github.com/ledgerwatch/erigon/p2p.(*dialTask).dial(0xc01a08e480, 0xc002608580, 0xc06b8c5440)
         github.com/ledgerwatch/erigon/p2p/dial.go:545 +0x67
 github.com/ledgerwatch/erigon/p2p.(*dialTask).run(0xc01a08e480, 0x0?)
         github.com/ledgerwatch/erigon/p2p/dial.go:495 +0x6b
 github.com/ledgerwatch/erigon/p2p.(*dialScheduler).startDial.func1()
         github.com/ledgerwatch/erigon/p2p/dial.go:466 +0x51
 created by github.com/ledgerwatch/erigon/p2p.(*dialScheduler).startDial
         github.com/ledgerwatch/erigon/p2p/dial.go:464 +0x3f2
 goroutine 78930 [select]:
 net.(*netFD).connect.func2()
         net/fd_unix.go:118 +0x86
 created by net.(*netFD).connect
         net/fd_unix.go:117 +0x37b
 goroutine 78697 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78967 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 goroutine 78977 [chan receive]:
 github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo.func1()
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:73 +0x51
 created by github.com/ledgerwatch/erigon/cmd/sentry/sentry.NewPeerInfo
         github.com/ledgerwatch/erigon/cmd/sentry/sentry/sentry_grpc_server.go:72 +0x190
 erigon.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
 erigon.service: Failed with result 'exit-code'.
 erigon.service: Scheduled restart job, restart counter is at 1.
 Stopped erigon.
 Started erigon.

quickchase avatar May 19 '22 14:05 quickchase

can confirm. same here

froid1911 avatar May 20 '22 08:05 froid1911

@quickchase it's end of error message. need start of message - there is reason of error and where it happened. try on latest devel plz.

AskAlexSharov avatar May 26 '22 08:05 AskAlexSharov

FWIW I received a similar error when I ran out of disk space. Make sure you're running this against a large drive

wup-one avatar Aug 23 '22 18:08 wup-one

@quickchase please re-open if this is still an issue, and add some more log context if you can - looks as @wup-one said to possibly be a diskspace issue.

revitteth avatar Oct 20 '22 15:10 revitteth