bitcoin-sv Node stuck at block height: 754082

This is in continuation to my previous post - https://github.com/bitcoin-sv/bitcoin-sv/issues/269

Our node is again stuck at block height: 754082 from almost two weeks:

26 Aug - 10am: 754082/754345
29 Aug - 06pm: 754082/754808
30 Aug - 02pm: 754082/754947
01 Sep - 10am: 754082/755194
02 Sep - 11am: 754082/755347
05 Sep - 10am: 754082/755780
06 Sep - 10am: 754082/755939

Here is bitcoind.log output:

2022-09-05 23:23:42 [msghand] New outbound peer connected: version: 70016, blocks=755884, peer=1500
2022-09-05 23:43:02 [msghand] New outbound peer connected: version: 70015, blocks=756533, peer=1502
2022-09-05 23:43:15 [msghand] New outbound peer connected: version: 70016, blocks=755892, peer=1503
2022-09-06 00:04:43 [msghand] New outbound peer connected: version: 80003, blocks=609135, peer=1506
2022-09-06 00:04:43 [msghand] New outbound peer connected: version: 70016, blocks=755895, peer=1507
2022-09-06 00:26:04 [msghand] New outbound peer connected: version: 70016, blocks=755899, peer=1511
2022-09-06 00:26:17 [msghand] New outbound peer connected: version: 70016, blocks=755899, peer=1513
2022-09-06 00:47:43 [msghand] New outbound peer connected: version: 70016, blocks=755902, peer=1515
2022-09-06 01:01:20 [msghand] Misbehaving: 77.120.122.15:8333 peer=1517 (0 -> 100) reason: invalid-UA BAN THRESHOLD EXCEEDED
2022-09-06 01:08:49 [msghand] New outbound peer connected: version: 70016, blocks=755903, peer=1518
2022-09-06 01:28:47 [msghand] New outbound peer connected: version: 70016, blocks=755906, peer=1520
2022-09-06 01:49:49 [msghand] New outbound peer connected: version: 70016, blocks=755907, peer=1523
2022-09-06 02:01:28 [msghand] Misbehaving: 135.181.144.73:8333 peer=1525 (0 -> 100) reason: invalid-UA BAN THRESHOLD EXCEEDED
2022-09-06 02:10:35 [msghand] New outbound peer connected: version: 70016, blocks=755913, peer=1528
2022-09-06 02:32:06 [msghand] New outbound peer connected: version: 70016, blocks=755917, peer=1533
2022-09-06 02:52:22 [msghand] New outbound peer connected: version: 70016, blocks=755921, peer=1535
2022-09-06 02:52:31 [msghand] New outbound peer connected: version: 70015, blocks=756557, peer=1538
2022-09-06 02:52:32 [msghand] New outbound peer connected: version: 70016, blocks=755921, peer=1539
2022-09-06 02:52:40 [msghand] New outbound peer connected: version: 70016, blocks=755921, peer=1542
2022-09-06 03:12:58 [msghand] New outbound peer connected: version: 80003, blocks=756559, peer=1544
2022-09-06 03:12:59 [msghand] New outbound peer connected: version: 70016, blocks=755241, peer=1545
2022-09-06 03:28:17 [msghand] New outbound peer connected: version: 70016, blocks=755925, peer=1548
2022-09-06 03:34:54 [msghand] New outbound peer connected: version: 70016, blocks=755925, peer=1550
2022-09-06 03:39:24 [msghand] Misbehaving: 89.179.247.236:8333 peer=1552 (0 -> 100) reason: invalid-UA BAN THRESHOLD EXCEEDED
2022-09-06 03:56:18 [msghand] New outbound peer connected: version: 80003, blocks=756563, peer=1555
2022-09-06 03:56:32 [msghand] New outbound peer connected: version: 70015, blocks=756563, peer=1556
2022-09-06 03:56:46 [msghand] New outbound peer connected: version: 80003, blocks=740238, peer=1557
2022-09-06 03:56:49 [msghand] New outbound peer connected: version: 70016, blocks=755930, peer=1558
2022-09-06 04:16:39 [msghand] New outbound peer connected: version: 70016, blocks=755932, peer=1561
2022-09-06 04:18:07 [msghand] New outbound peer connected: version: 70016, blocks=755933, peer=1563
2022-09-06 04:38:44 [msghand] New outbound peer connected: version: 70016, blocks=755936, peer=1568
2022-09-06 05:00:30 [msghand] New outbound peer connected: version: 70016, blocks=755938, peer=1570
2022-09-06 05:21:15 [msghand] New outbound peer connected: version: 70016, blocks=755939, peer=1574
2022-09-06 05:21:18 [msghand] New outbound peer connected: version: 70016, blocks=755939, peer=1576
2022-09-06 05:43:03 [msghand] New outbound peer connected: version: 80003, blocks=756575, peer=1578
2022-09-06 05:43:11 [msghand] New outbound peer connected: version: 70016, blocks=755940, peer=1579
2022-09-06 06:04:15 [msghand] New outbound peer connected: version: 70016, blocks=755940, peer=1581
2022-09-06 06:04:43 [msghand] New outbound peer connected: version: 70016, blocks=755940, peer=1583
2022-09-06 06:13:28 [msghand] New outbound peer connected: version: 70016, blocks=755940, peer=1585
2022-09-06 06:26:34 [msghand] New outbound peer connected: version: 70016, blocks=755941, peer=1587
2022-09-06 06:46:47 [msghand] New outbound peer connected: version: 70016, blocks=755941, peer=1590
2022-09-06 06:47:13 [msghand] New outbound peer connected: version: 70016, blocks=755941, peer=1593
2022-09-06 06:47:27 [msghand] New outbound peer connected: version: 70016, blocks=755941, peer=1595
2022-09-06 07:08:56 [msghand] New outbound peer connected: version: 70016, blocks=755942, peer=1600
2022-09-06 07:27:26 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1604
2022-09-06 07:29:21 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1606
2022-09-06 07:29:23 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1608
2022-09-06 07:49:31 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1611
2022-09-06 08:09:53 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1613
2022-09-06 08:09:56 [msghand] New outbound peer connected: version: 70016, blocks=755944, peer=1615
2022-09-06 08:31:49 [msghand] New outbound peer connected: version: 70016, blocks=755945, peer=1619
2022-09-06 08:53:01 [msghand] New outbound peer connected: version: 70016, blocks=755946, peer=1622
2022-09-06 08:56:56 [msghand] Misbehaving: 104.155.81.197:8333 peer=1624 (0 -> 100) reason: invalid-UA BAN THRESHOLD EXCEEDED
2022-09-06 09:13:40 [msghand] New outbound peer connected: version: 70016, blocks=755946, peer=1627
2022-09-06 09:31:34 [msghand] New outbound peer connected: version: 70016, blocks=755946, peer=1630
2022-09-06 09:32:12 [msghand] New outbound peer connected: version: 70016, blocks=755946, peer=1634
2022-09-06 09:33:41 [msghand] New outbound peer connected: version: 80003, blocks=756597, peer=1636
2022-09-06 09:33:42 [msghand] New outbound peer connected: version: 80003, blocks=756597, peer=1637
2022-09-06 09:33:55 [msghand] New outbound peer connected: version: 70016, blocks=755252, peer=1638

Output of getnetworkinfo:

{
  "version": 101001100,
  "subversion": "/Bitcoin SV:1.0.11/",
  "protocolversion": 70016,
  "localservices": "0000000000000021",
  "localrelay": true,
  "timeoffset": 0,
  "txnpropagationfreq": 250,
  "txnpropagationqlen": 0,
  "networkactive": true,
  "connections": 8,
  "addresscount": 33906,
  "streampolicies": "BlockPriority,Default",
  "networks": [
    {
      "name": "ipv4",
      "limited": false,
      "reachable": true,
      "proxy": "",
      "proxy_randomize_credentials": false
    },
    {
      "name": "ipv6",
      "limited": false,
      "reachable": true,
      "proxy": "",
      "proxy_randomize_credentials": false
    }
  ],
  "relayfee": 0.00000000,
  "minconsolidationfactor": 20,
  "maxconsolidationinputscriptsize": 150,
  "minconfconsolidationinput": 6,
  "minconsolidationinputmaturity": 6,
  "acceptnonstdconsolidationinput": false,
  "localaddresses": [
  ],
  "warnings": "Warning: The network does not appear to agree with the local blockchain! Still waiting for block data for more details. We are in the startup process (e.g. Initial block download or reindex), this might be reason for disagreement."
}

excessiveblocksize=200000000000

Any help pls.

Sep 06 '22 11:09 prashant-1926

You might want to try invalidating that block.

Sep 09 '22 00:09 cdluv

@cdluv How to invalidate a block? I'm unaware of doing it.. Can you pls help by elaborating it a bit more.

Sep 12 '22 06:09 prashant-1926

You'll need to look at the RPC API. "getnetworkinfo" is one such call, so you're already using the API. It will tell you how to invalidate a block - and the call is probably "invalidateblock"? I might be wrong! IIRC, as you peer is stuck on a block, as it was likely to be syncing with a competing chain that no longer exists, and the hashes of the subsequent block, don't make sense to the peer.

By invalidating that block, your peer might be able to move forward.

If you're still stuck, have a look here for further insights: https://github.com/Blockchair/Blockchair.Support/issues/910

Sep 13 '22 23:09 cdluv

Thank you for the inputs. I tried with invalidateblock but of no use, node was still stuck at the same block 754081 for many days. Now, getting a clue from second link you have shared, I'm re-indexing the blocks, hope it may help solve the problem. Right now it is at the the tip 724939/753222.

Sep 16 '22 11:09 prashant-1926

invalidateblock is not meant for this purpose. This issue is often related to inadequate hardware. See also this article in the Knowledge Base: Unable to complete Initial Block Download (IBD)

Sep 20 '22 12:09 pfromberg

Thank you for the inputs. I tried with invalidateblock but of no use, node was still stuck at the same block 754081 for many days. Now, getting a clue from second link you have shared, I'm re-indexing the blocks, hope it may help solve the problem. Right now it is at the the tip 724939/753222.

if anyone searches for this block, the next one is over 2 gigs so you may have set the limit by one zero too few

Nov 14 '23 15:11 iopq