avalanchego
avalanchego copied to clipboard
Published
20 hours ago •
ava-labs
ava-labs
v1.7.12 database corruption during bootstrap
Jun 05 01:06:00 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:00.402] INFO <X Chain> bootstrap/bootstrapper.go:264 Starting bootstrap...
Jun 05 01:06:00 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:00.402] INFO <P Chain> bootstrap/bootstrapper.go:530 waiting for the remaining chains in this subnet to finish syncing
Jun 05 01:06:00 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:00.442] INFO server/server.go:238 adding route /ext/bc/2oYMBNV4eNHyqk2fjjV5nVQLDbtmNJzq5s3qs3Lo6ftnC6FByM
Jun 05 01:06:00 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:00.442] INFO server/server.go:238 adding route /ext/bc/2oYMBNV4eNHyqk2fjjV5nVQLDbtmNJzq5s3qs3Lo6ftnC6FByM/wallet
Jun 05 01:06:00 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:00.442] INFO server/server.go:238 adding route /ext/bc/2oYMBNV4eNHyqk2fjjV5nVQLDbtmNJzq5s3qs3Lo6ftnC6FByM/events
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.917] INFO <X Chain> common/bootstrapper.go:224 Bootstrapping started syncing with 1 vertices in the accepted frontier
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.917] INFO <X Chain> bootstrap/bootstrapper.go:477 bootstrapping fetched 0 vertices. Executing transaction state transitions...
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.917] INFO <X Chain> queue/jobs.go:182 executed 0 operations
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.917] INFO <X Chain> bootstrap/bootstrapper.go:488 executing vertex state transitions...
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.918] INFO <X Chain> queue/jobs.go:182 executed 0 operations
Jun 05 01:06:14 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:14.918] INFO <X Chain> bootstrap/bootstrapper.go:516 waiting for the remaining chains in this subnet to finish syncing
Jun 05 01:06:15 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:15.408] INFO <C Chain> common/bootstrapper.go:224 Bootstrapping started syncing with 1 vertices in the accepted frontier
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.261] INFO <C Chain> bootstrap/bootstrapper.go:491 bootstrapping fetched 2491766 blocks. Executing state transitions...
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: ERROR[06-05|01:06:16.393] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:1291:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: ########## BAD BLOCK #########
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: Chain config: {ChainID: 43114 Homestead: 0 DAO: 0 DAOSupport: true EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Apricot Phase 1: 1617199200, Apricot Phase 2: 1620644400, Apricot Phase 3: 1629813600, Apricot Phase 4: 1632344400, Apricot Phase 5: 1638468000, Engine: Dummy Consensus Engine}
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: Number: 13119518
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: Hash: 0x5f97b63c5c2a13f72a55792ac98d67e811ade5c200fb1f8925db4eb59c2e7302
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 0: cumulative: 108004 gas: 108004 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8970e25554551b69992ca32181ea3c04d1e7e9975f53863af151c73421bf5823 logs: [0xc021f969a0 0xc021f96a50 0xc021f96b00] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000800000000000000000000008100000001000000000040000000000000040000000000000020000000000000100000800000000000000080010000010000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000008000000012000000080000000000000000000000000004000000000000080030000000000800000000000000000000000000000000000000000000000000008000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 1: cumulative: 294519 gas: 186515 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb13d9580f6c9dcd44087be742f09e5a72e677120e266e64d7a158b7e9c96c684 logs: [0xc0216f1970] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 2: cumulative: 540812 gas: 246293 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x176882f29bfed70cba3cc95f4201f7f0419a241e7f701ceb9543a7b5cc5174ec logs: [0xc0216f1a20 0xc0216f1ad0 0xc0216f1b80 0xc0216f1c30] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020000000000000100000800000000000000000010000010000000000000000000000000000000080000000000000000000000000000000000200000000000000200000000000000200000000000200800000000000001200000000000000012000000080000000000000000000000000004000000000000000030000000100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 3: cumulative: 783295 gas: 242483 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x3ab8d4713f403a15371b40e75eeff97f6681fc5671a0796f00e243d876e0270b logs: [0xc021f96fd0 0xc021f97080 0xc021f97130 0xc021f971e0] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020008000000000100000800000000000000000010000010000000000000000000000000000000000000000000000000000000000000000000200000000000000200000000000000200000000000000800000000000001000040000000000012000000080000000000000000000000000004000000000000000030000100100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 4: cumulative: 856699 gas: 73404 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x6c556de9f13c001bfac7c744a81185942da6d4c340760a680484ac3fe06c72e0 logs: [0xc021f97290] bloom: 00000000020000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000020000000000000004000000000000000000000000000000020000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000004000008000000000020000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000080000000800000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 5: cumulative: 1016846 gas: 160147 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc56ac13ae2b9e9653ad967976d388eaba3a28b8cc4aa725ce6f0cc1194838bc5 logs: [0xc021f974a0] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 6: cumulative: 1203390 gas: 186544 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x9c4368026be0362bf9b1cab93c0c357629479b51bf64b0b3d60759d9a5b503a6 logs: [0xc021f97550] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 7: cumulative: 1436980 gas: 233590 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x9740cfbdb23db86f873897689c372e5911110dd2f8b87cc8ce6b2dbbedd5314c logs: [0xc021f97600 0xc021f976b0 0xc021f97760 0xc021f97810] bloom: 00000000000000000000000000000000000000000000000400000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020000000000000100000800000000000000000010000010000000000000000000000000000000000000000000000000000000000000000000200000000000000200000000000000200000002000000800000000000001000000000000000012000000080000000000000000000000000004000000010000000030000000100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 8: cumulative: 1645533 gas: 208553 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x700d18dbcd266785c993061d2a6234a5047b955aba48a5684543eeafd61d128a logs: [0xc021f978c0 0xc021f97970 0xc021f97a20 0xc021f97ad0 0xc021f97b80 0xc021f97c30] bloom: 00000000000000000000000000000000000000002000000000000010080000000000000000000000000000000000400001000000000000000000000000200000000000000008000000000008100000000000000000000000000000008000000000000000000000000000000100000000000000000000000010000010000000000000002000000000000000000000000000000000000000000000000000000000020000004200000000000000000000000000000000000000010001000000000000000052000000080000000000000000000080000004000000000000000010000010000000400000000000000000000000000000000000010000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 9: cumulative: 1832270 gas: 186737 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xf8f23e032f72c8984bd101b0b7cb6aaa6fe8b8f96336f7dda4e9001109a870f9 logs: [0xc021f97ce0] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 10: cumulative: 2046590 gas: 214320 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xac3f2388811ea3b3c8a1334fd1b70cdf992ce827623eef9cc50f430cff17bc47 logs: [0xc023b122c0] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000050000000000000000000000000000000000010000000000000400000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 11: cumulative: 2154594 gas: 108004 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x17589e15093e5defad6b8e6e125df4b256ade73d4a73cda629e9aafec3631467 logs: [0xc023b12370 0xc023b12420 0xc023b124d0] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000800000000000000000000008100000001000800000040000000000000000000000000000020000000000000100002800000000000000080010000010000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000400000000000000000000000000001000000000000000012000000080000000000000000000000000004000000000000000030000000000800000000000000000000000000000000000000000000000000008000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 12: cumulative: 2215732 gas: 61138 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x16c5bcf55394716fa992d2ae07a6bf561971b1fc6f90c0f5c1a7f67357eeca1f logs: [0xc023b12580] bloom: 00000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000002000000000000000008000000000000000000000000000000000000000010000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000004000000000000000040000008000000000020000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 13: cumulative: 2292384 gas: 76652 contract: 0x0000000000000000000000000000000000000000 status: 0 tx: 0xd261b5b18e6fea0f805a1c69cbdc959e0dd4aa83701839432102329c3ee832af logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 14: cumulative: 2338890 gas: 46506 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x670dadbb93116f8a209561f615061d3831a9907a0e14622e9427939c7070700b logs: [0xc023b12790] bloom: 00000000000000000010000000000000000000000800000000000000000000000000000800000040000000000000000000000000000000400000000000a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000080000000400000000020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 15: cumulative: 3367087 gas: 1028197 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc38fd0d0ee3c30fea1943c663605a7f806f15375c86dc41a007d16813fa18b02 logs: [0xc023b128f0 0xc023b129a0 0xc023b12a50 0xc023b12b00 0xc023b12bb0 0xc023b12c60 0xc023b12d10 0xc023b12dc0 0xc023b12e70 0xc023b12f20 0xc023b12fd0 0xc023b13080 0xc023b13130 0xc023b131e0] bloom: 00200000000000000001002080000000000000000800000020000000000000001040000000000040004000000000000000000000000000000000000000800000002080000200000000022008400000200000000000400000000000000000000000000000020004008000000000000810000000000000040000000010000000000000000000000000000000000000000000000020100000080010004000000000000000000000040000000000000000000000000000000000000000080000000000008002000000000000000000020000042000000000001000000002000024000000000000000000000000000000000800000000000000200000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 16: cumulative: 4013955 gas: 646868 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xa17e9fb978f0f260bcca03478791bd0f763703342897b1c9839d51071b3f5d01 logs: [0xc023a8c790 0xc023a8c840 0xc023a8c8f0 0xc023a8c9a0 0xc023a8ca50 0xc023a8cb00 0xc023a8cbb0 0xc023a8cc60 0xc023a8cd10 0xc023a8cdc0 0xc023a8ce70 0xc023a8cf20 0xc023a8cfd0 0xc023a8d080] bloom: 00200000000000000001002080001000000000000800000020000000000000000040000000000040024000000000000000000000000000000000000000800000002080000200000000022008400000200000000000400000000000000000000000000000020004000000000000000800000000000000040000000010000000000000000000000000000000000000000000000020100000080000004000000000000000000000040000000200000000000000000000000000000000080000000000008002000000000000000000020000002000000002001000000022000020000000000000000000000000000000000800000000000000202000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 17: cumulative: 4109131 gas: 95176 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x93c0e831d374028d462d020553ac22a1a302c0902d9bd718b8adc9c1da5d0e11 logs: [0xc024188210 0xc0241882c0 0xc024188370 0xc024188420 0xc0241884d0] bloom: 4000000000000000000000000000000000000000000020000000808000100000000400000000000000000000000000000000000000100000000010000021000000000000000000000008000800000000000000000000000000000000000004000000000002000000000000000000080000000000000410000000001000200000000000000000000000000000000000000001000000000000000000000000001002000000000000000000000000000000000000000008000000000000000000000000000200000000000000000000000000000000000000000000000000002000001000000000000000000000000000000c000000000002000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 18: cumulative: 4355571 gas: 246440 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8c146561ccaa3823811a17bbfc5be38182563fbc0d7729fb092ed2c16612375b logs: [0xc023a8d130 0xc023a8d1e0 0xc023a8d290 0xc023a8d340] bloom: 00000004000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000800000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020000000000000100000800000000000000000010000010000000000000000000000000000000000000000000000000000000000000000000200000000000000200000000000000200000000000000800000000000001000000000000000012000000080000000000000000000000000004000000000200000030000000100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 19: cumulative: 4564112 gas: 208541 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8b29d7f27b52aeeddb28e1ea4c3e520ab0b41cecf7acdc209dffaf74019afffa logs: [0xc024188630 0xc0241886e0 0xc024188790 0xc024188840 0xc0241888f0 0xc0241889a0] bloom: 00000000000000000000000000000000000001002400000000000000000000080000000000000000000000000000400800000000000000000000000000200000000000000008000000000008100000000000000000000000000000008000000000000000000000000000000100000000000000000000000010000010000000000000002000000000000000000000000000000000000000000000000000000000020000004200000000000000000000000000000000000000000001000000000400000012000000080000000000000000000080000004000000000000000010000010000000000000000000000000000000000000000000010800000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 20: cumulative: 4778432 gas: 214320 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xd694ba424a3112e6214c510cf9507fe217ed052658a28b7d0fb84259684a2b22 logs: [0xc024188b00] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000050000000000000000000000000000000000010000000000000400000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 21: cumulative: 5022690 gas: 244258 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xcff082ea0752b84905d30389fb14f2797d25c34719ee5941df46ee0c9b34c734 logs: [0xc023a8d550 0xc023a8d600 0xc023a8d6b0 0xc023a8d760] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000008000000000008000000000000000000000000400000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020000000000000100000800000000000000000010000010000000000000000000000000000000000000000000000000000000000000000000200000000000000200000000000000200000000000000800000000200001000000000000000012000000080000000000000000000000000004000000000000000030000000100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 22: cumulative: 5266848 gas: 244158 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb921e0c183f78e1852363c663b6da84868a917fc67084d7143faa0cc3844ce28 logs: [0xc023a8d810 0xc023a8d8c0 0xc023a8d970 0xc023a8da20] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000008100000000000000000040000000000000000040000000000020000000000000100000800000000000000000010000010000000000200000000000000080000000000002000000000000000000000000000200000000000000200000000000000200000000000000800000000000001000000000000000012000000080000000000000000000000000004000000000000000030000000100800000000000200040000000000000000000000000000000000108000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 23: cumulative: 5475413 gas: 208565 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x0ca75c48ebe46908cc8e7682ab3f11908480f49d3d68c96f174e674f04d06a77 logs: [0xc023a8dad0 0xc023a8db80 0xc023a8dc30 0xc023a8dce0 0xc023a8dd90 0xc023a8de40] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000400000000000000000000000000000200000000000000008000000000008100000000000000000000000000000008000000000000000000000000000000100000000000000000000000010010010000000000040002000000000000000002000000000000000000000000000000000000000220000004200000000000000000000800000000000000000000001000002000000000012000000080000000000000000000080000004000000000000000010000010000000000000000000000000000000000000000000010000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 24: cumulative: 5683966 gas: 208553 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x478edbbeade27bebbbbd745a6fc6af87952d50f3d2bb9182fbf468ce4a75450d logs: [0xc0234b2dc0 0xc0234b2e70 0xc0234b2f20 0xc0234b2fd0 0xc0234b3080 0xc0234b3130] bloom: 00000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000400000000000000000000000000000200000000000000008000000000008100000000000000000000000000000008000000000000000000000000000000100000000000000000000000010000010000000000400002000008000000000000000000000000000000004000000000000000000020000004200000400000000000000000000000800000000000001000000000000000012000000080000020000000000000080000004000000000000000010000010000000000000000000000000000000000000000000010000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 25: cumulative: 5707795 gas: 23829 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x33f4446fdc2ee53d3e095bfcb1754ac852ff8e8f3402b7056a1f24e7910786eb logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: 26: cumulative: 5917514 gas: 209719 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x0b241c535a597cd1daed50df8854f0bc9abcfaa1d0ad8413fff2ac31bb87dd00 logs: [0xc023a8def0] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000001000000000000000050000000000000000000000000000000000010000000000000400000000000000000000000000000000000000000000000000000000000000000000000 state:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: Error: invalid merkle root (remote: 401fa843b625fe9e9171011eb10b8316f03322c8e7ec5426b5479afce0b34886 local: dc76dcfa12e25830bf26aefdd5ddcc76cf3038aae5978da12495b291e0c1d87e)
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: ##############################
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]:
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.394] ERROR <C Chain> bootstrap/block_job.go:86 block 9ipm8r79TsVVAG96pYPLub8g39n5GeGBxPrQCRejrBsUSMH3W failed verification during bootstrapping due to invalid merkle root (remote: 401fa843b625fe9e9171011eb10b8316f03322c8e7ec5426b5479afce0b34886 local: dc76dcfa12e25830bf26aefdd5ddcc76cf3038aae5978da12495b291e0c1d87e)
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] FATAL <C Chain> handler/handler.go:274 shutting down chain due to unexpected error: failed to execute job 9ipm8r79TsVVAG96pYPLub8g39n5GeGBxPrQCRejrBsUSMH3W due to failed to verify block in bootstrapping: invalid merkle root (remote: 401fa843b625fe9e9171011eb10b8316f03322c8e7ec5426b5479afce0b34886 local: dc76dcfa12e25830bf26aefdd5ddcc76cf3038aae5978da12495b291e0c1d87e) while processing sync message: (Op: ancestors, NodeID: NodeID-HPWAaJ3w4JwWoeFpfbjDB6YnYTDX7fRJq, RequestID: 3, NumContainers: 415)
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO <C Chain> bootstrap/bootstrapper.go:256 shutting down bootstrapper
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/tx_pool.go:457: Transaction pool stopped
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:688: Stopping Acceptor
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:691: Acceptor queue drained t="9.507µs"
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:693: Shutting down state manager
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:698: State manager shut down t="1.247µs"
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:701: Shutting down sender cacher
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:705: Closing scope
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: INFO [06-05|01:06:16.395] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:708: Blockchain stopped
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO node/node.go:1281 shutting down node with exit code 1
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO ipcs/chainipc.go:101 shutting down chain IPCs
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO chains/manager.go:974 shutting down chain manager
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO router/chain_router.go:252 shutting down chain router
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO <P Chain> bootstrap/bootstrapper.go:256 shutting down bootstrapper
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.395] INFO <X Chain> bootstrap/bootstrapper.go:257 shutting down bootstrapper
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.396] INFO network/network.go:1021 shutting down the p2p networking
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.490] INFO node/node.go:1324 cleaning up plugin subprocesses
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.498] INFO node/node.go:1334 finished node shutdown
Jun 05 01:06:16 ip-10-0-0-131 avalanchego[162600]: [06-05|01:06:16.517] INFO nat/nat.go:152 Unmapped all ports
Jun 05 01:06:16 ip-10-0-0-131 systemd[1]: avalanchego.service: Main process exited, code=exited, status=1/FAILURE
Describe the bug Database corruption, stuck in a restart loop with this bad block being processed repeatedly
To Reproduce I updated to v1.7.12 during an unfinished bootstrap phase from v1.7.11 but i believe this can be caused generally by restarts during the bootstrap phase
Expected behavior Clearing the bad block or a recovery option
Operating System recent ubuntu server
Additional context I am not blocked by this, i have deleted the db directory and will start bootstrap from fresh - it would be nice to have a recovery path though
The same problem. Tried to sync two times, having this 'bad block' error at the end of the sync.
Chain config: {ChainID: 43114 Homestead: 0 DAO: 0 DAOSupport: true EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Apricot Phase 1: 1617199200, Apricot Phase 2: 1620644400, Apricot Phase 3: 1629813600, Apricot Phase 4: 1632344400, Apricot Phase 5: 1638468000, Engine: Dummy Consensus Engine}
Number: 1602104
Hash: 0x9b2002c2e4a6ac1853c5fd035ebdd0cc0c32ed3e4326edd5acb84ed803886a75
0: cumulative: 741140 gas: 741140 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x2aa2cca4214772e72c73917b0ac9e3523bc1c074a38e1193ab75e77d475681eb logs: [0xc01b884210 0xc01b8842c0 0xc01b884370 0xc01b884420 0xc01b8844d0] bloom: 00200000000000000000000080000000000000000002000000000000000000000000000000000000000000000000000000000000000000000080000000200200000000000000020000000008000000200000000020000000000000000000000000000200000000000000000000000000000000000002000800000010000000000000000000000000000000000000002000000000000000080000004000000080020000000000000100000000000000000000000000000000000000020000000000000002000000000000000000000000000000004000001000000002000000000010000000000000000000000022000000004000000000000000000001000000 state:
1: cumulative: 905997 gas: 164857 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xd8afb03089aafe5d30be489e3d80ee0169dc20014d8c4f1e54e5d5aad2ce141c logs: [0xc01b884840 0xc01b8848f0 0xc01b8849a0 0xc01b884a50 0xc01b884b00 0xc01b884bb0 0xc01b884c60] bloom: 00000000000000000000400280000000000400008000000000000000000001000000004000000000000000000000000080000000000000000000000000208000000000000000000000000008000000000000000000000020000000000000000000000000220000000000000000000840000000000008000000000010000000004000000400000100000000800000000000000000000000080000000008000000020000000000000000000000000000000000000000000000000000000000000000000002000000000000000000020000000000000000001000002000000020000010000000000020000000000800000000000000000000000000000000000000 state:
2: cumulative: 950439 gas: 44442 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x308c2f53f442404e79d2250a2e150eb13ab235112584a3cd0890e8d363aec98f logs: [0xc01b884f20] bloom: 00000000000000000000000000040000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000210000000000000000800000000000000000000000000000000000000000000000000001000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000080000000000000080000000000000000000000000000000020000000000000000010000000000000000000000000000000000000000000000000000000000000 state:
3: cumulative: 1051832 gas: 101393 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xfeeda2a043c5012c0221a5eeb621c9d266a62664e8c0a502ae7ded19d08e3fb3 logs: [0xc017cfb4a0 0xc017cfb550 0xc017cfb600 0xc017cfb6b0] bloom: 00000000000000000000000000000000000002100000000000004000000000000000020002000200000000080000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000010000000000004000000000000000400000000000000000000000000000000000000000810000000000000200200000000000000000000000000000000000000000000000000000002401000000000400000000000000000000000200000000100010000200000040000000000000000002020000000000000000000000000000000000000 state:
Error: invalid merkle root (remote: 9635c68a10b897dd5007a2ba3b305424abfe65415b6ca63a7731df3e8c4bf77d local: 6679d3f2c0f74574e6f2ca73fd0ad986cbe9be4e4896c4b0c226000fa1ded71c)
##############################
[07-06|14:38:54.823] ERROR <C Chain> bootstrap/block_job.go:86 block 2BKTjovWc2jjjPmqAC6n9kiWRJ25H3YaA9EeHPNSK5QAcBFGCV failed verification during bootstrapping due to invalid merkle root (remote: 9635c68a10b897dd5007a2ba3b305424abfe65415b6ca63a7731df3e8c4bf77d local: 6679d3f2c0f74574e6f2ca73fd0ad986cbe9be4e4896c4b0c226000fa1ded71c)
[07-06|14:38:54.823] FATAL <C Chain> handler/handler.go:276 shutting down chain due to unexpected error: failed to execute job 2BKTjovWc2jjjPmqAC6n9kiWRJ25H3YaA9EeHPNSK5QAcBFGCV due to failed to verify block in bootstrapping: invalid merkle root (remote: 9635c68a10b897dd5007a2ba3b305424abfe65415b6ca63a7731df3e8c4bf77d local: 6679d3f2c0f74574e6f2ca73fd0ad986cbe9be4e4896c4b0c226000fa1ded71c) while processing sync message: (Op: ancestors, NodeID: NodeID-LqDSTXYTRe7fWsZNUhtVaTUYhCQhwm4mj, RequestID: 24, NumContainers: 294)
[07-06|14:38:54.827] INFO <C Chain> bootstrap/bootstrapper.go:258 shutting down bootstrapper
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/tx_pool.go:457: Transaction pool stopped
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:688: Stopping Acceptor
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:691: Acceptor queue drained t="9.367µs"
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:693: Shutting down state manager
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:698: State manager shut down t="12.483µs"
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:701: Shutting down sender cacher
INFO [07-06|14:38:54.827] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:705: Closing scope
INFO [07-06|14:38:54.828] <C Chain> github.com/ava-labs/coreth/core/blockchain.go:708: Blockchain stopped
[07-06|14:38:54.828] INFO node/node.go:1290 shutting down node with exit code 1
[07-06|14:38:54.828] INFO ipcs/chainipc.go:101 shutting down chain IPCs
[07-06|14:38:54.828] INFO chains/manager.go:975 shutting down chain manager
[07-06|14:38:54.828] INFO router/chain_router.go:265 shutting down chain router
[07-06|14:38:54.828] INFO <X Chain> bootstrap/bootstrapper.go:257 shutting down bootstrapper
[07-06|14:38:54.828] INFO <P Chain> bootstrap/bootstrapper.go:258 shutting down bootstrapper
[07-06|14:38:54.828] INFO network/network.go:1030 shutting down the p2p networking
[07-06|14:38:54.874] INFO node/node.go:1333 cleaning up plugin subprocesses
[07-06|14:38:54.893] INFO node/node.go:1343 finished node shutdown
[07-06|14:38:54.893] INFO nat/nat.go:152 Unmapped all ports
OS: Ubuntu 20.04 Avalanchego: avalanchego-v1.7.14 Any advices on how to avoid this?
Also, I checked the logs. The initial error before restart was:
[07-06|09:51:17.169] ERROR <C Chain> bootstrap/block_job.go:86 block pB197WHBA894KgjWFgk9SEvVTFxepeK8xj9D2NDjQpFYjibzo failed verification during bootstrapping due to missing trie node 92ca18a03ae8445aee>
<ify block in bootstrapping: missing trie node 92ca18a03ae8445aee7da883be036da4489b6ae7867c7756d5b251cb41b150d0 (path 01) while processing sync message: (Op: ancestors, NodeID: NodeID-AoHdNKxXdndkKrbQegn>
[07-06|09:51:17.171] INFO <C Chain> bootstrap/bootstrapper.go:258 shutting down bootstrapper
@aphexmunky do you have a database that is currently broken we could use to repro?
@patrick-ogrady I don't have a copy of a broken one sorry. Happy to close this issue and if it happens again i'll clone and reach out to you.
