Stack overflow on sync after hard restart

[skillful-alex]

System information

erigon version 2022.08.2-alpha-35c4faa1 Ubuntu 20.04.4 LTS

Actual behaviour

fatal error: stack overflow

Steps to reproduce the behaviour

hard stop sync and then start it again ./erigon --prune htc --datadir=/blockchain --chain=mainnet --metrics --private.api.addr=127.0.0.1:9091

Backtrace

INFO[08-18|12:18:16.995] Starting metrics server                  addr=http://127.0.0.1:6060/debug/metrics/prometheus
INFO[08-18|12:18:16.995] Build info                               git_branch=HEAD git_tag=v2022.08.02 git_commit=35c4faa1b41e8379a74d0385505add0dd450c2ed
INFO[08-18|12:18:16.995] Starting Erigon on Ethereum mainnet... 
INFO[08-18|12:18:16.998] Maximum peer count                       ETH=100 total=100
INFO[08-18|12:18:16.998] starting HTTP APIs                       APIs=eth,erigon,engine
INFO[08-18|12:18:16.998] torrent verbosity                        level=WRN
INFO[08-18|12:18:19.101] Set global gas cap                       cap=50000000
INFO[08-18|12:18:19.200] Opening Database                         label=chaindata path=/blockchain/chaindata
INFO[08-18|12:18:19.204] Initialised chain configuration          config="{ChainID: 1, Homestead: 1150000, DAO: 1920000, DAO Support: true, Tangerine Whistle: 2463000, Spurious Dragon: 2675000, Byzantium: 4370000, Constantinople: 7280000, Petersburg: 7280000, Istanbul: 9069000, Muir Glacier: 9200000, Berlin: 12244000, London: 12965000, Arrow Glacier: 13773000, Gray Glacier: 15050000, Terminal Total Difficulty: 58750000000000000000000, Merge Netsplit: <nil>, Engine: ethash}" genesis=0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3
INFO[08-18|12:18:19.204] Effective                                prune_flags="--prune.h.older=90000 --prune.t.older=90000 --prune.c.older=90000" snapshot_flags="--snapshots=true" history.v2=false
WARN[08-18|12:18:19.234] [snapshots] open segment                 err="invalid snapshot format. decompress.buildPatternTable faced maxDepth underflow, fileName: v1-002500-003000-headers.seg"
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc034e00378 stack=[0xc034e00000, 0xc054e00000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x19a9027?, 0x2ba5480?})
	runtime/panic.go:992 +0x71
runtime.newstack()
	runtime/stack.go:1101 +0x5cc
runtime.morestack()
	runtime/asm_amd64.s:547 +0x8b

goroutine 1 [running]:
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06616e0e0, {0xc003ad7b78?, 0xea7e7?, 0x100091?}, {0xc004301268?, 0xea7e7, 0x118891?}, 0x0, 0x9, 0x24dcfe, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:179 +0x639 fp=0xc034e00388 sp=0xc034e00380 pc=0xe6a799
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06616e0e0, {0xc003ad7b78?, 0xea7e7, 0x100091}, {0xc004301268?, 0xea7e7, 0x118891}, 0x0, 0x8, 0x24dcfd, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:222 +0x399 fp=0xc034e00450 sp=0xc034e00388 pc=0xe6a4f9
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06616e0e0, {0xc003ad7b78?, 0xea7e7, 0x100091}, {0xc004301268?, 0xea7e7, 0x118891}, 0x0, 0x7, 0x24dcfc, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:222 +0x399 fp=0xc034e00518 sp=0xc034e00450 pc=0xe6a4f9
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06616e0e0, {0xc003ad7b78?, 0xea7e7, 0x100091}, {0xc004301268?, 0xea7e7, 0x118891}, 0x0, 0x6, 0x24dcfb, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:222 +0x399 fp=0xc034e005e0 sp=0xc034e00518 pc=0xe6a4f9
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06616e0e0, {0xc003ad7b78?, 0xea7e7, 0x100091}, {0xc004301268?, 0xea7e7, 0x118891}, 0x0, 0x5, 0x24dcfa, ...)
................................................

[AskAlexSharov]

fixed by https://github.com/ledgerwatch/erigon/pull/4882 update if can.

if can't update: you may try to start erigon with —downloader.verify if doesn't help - you may try to start with --http=false if nothing helps - try: ./build/bin/downloader --datadir=/blockchain

[skillful-alex]

updated, error persisted

./erigon --prune htc --datadir=/blockchain --chain=mainnet --metrics --private.api.addr=127.0.0.1:9091
INFO[08-19|08:48:46.413] Starting metrics server                  addr=http://127.0.0.1:6060/debug/metrics/prometheus
INFO[08-19|08:48:46.413] Build info                               git_branch=devel git_tag=v2021.10.03-1709-g30324fbb8 git_commit=30324fbb815be0d6648e0d60ea3c9f2bfbe0015c
.....................................
INFO[08-19|08:48:48.624] Effective                                prune_flags="--prune.h.older=90000 --prune.t.older=90000 --prune.c.older=90000" snapshot_flags="--snapshots=true" history.v2=false
WARN[08-19|08:48:48.651] [snapshots] open segment                 err="invalid snapshot format. decompress.buildPatternTable faced maxDepth underflow, fileName: v1-002500-003000-headers.seg"
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc034e003d8 stack=[0xc034e00000, 0xc054e00000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x1996ccb?, 0x2b8b500?})
	runtime/panic.go:992 +0x71
runtime.newstack()
	runtime/stack.go:1101 +0x5cc
runtime.morestack()
	runtime/asm_amd64.s:547 +0x8b

goroutine 1 [running]:
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06604f080, {0xc0029d1b78?, 0xea7e7?, 0x100091?}, {0xc0073f7268?, 0xea7e7, 0x118891?}, 0x0, 0x5, 0x24dcfa, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:179 +0x639 fp=0xc034e003e8 sp=0xc034e003e0 pc=0xe6a779
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06604f080, {0xc0029d1b78?, 0xea7e7, 0x100091}, {0xc0073f7268?, 0xea7e7, 0x118891}, 0x0, 0x4, 0x24dcf9, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:222 +0x399 fp=0xc034e004b0 sp=0xc034e003e8 pc=0xe6a4d9
github.com/ledgerwatch/erigon-lib/compress.buildPatternTable(0xc06604f080, {0xc0029d1b78?, 0xea7e7, 0x100091}, {0xc0073f7268?, 0xea7e7, 0x118891}, 0x0, 0x3, 0x24dcf8, ...)
	github.com/ledgerwatch/[email protected]/compress/decompress.go:222 +0x399 fp=0xc034e00578 sp=0xc034e004b0 pc=0xe6a4d9
.....................................

[AskAlexSharov]

@awskii hi. plz help to:

• add file name to such panic
• make decompressor more resistant to bad files (need return error instead of panic even if file is really bad )

[github-actions[bot]]

This issue is stale because it has been open for 40 days with no activity. Remove stale label or comment, or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.