chia-blockchain icon indicating copy to clipboard operation
chia-blockchain copied to clipboard
verified publisher icon Chia-Network

[Bug] Generating K-32 plot had status Not Available for a long time and Full Node kept reverting to Not Synced

Open chia-rschmidt opened this issue 1 year ago • 4 comments

What happened?

After the K-32 plot finished being created, it reverted to Not Available for a long time and Full Node kept showing Not Synced instead of Syncing. After restarting the GUI, Full Node started syncing again slowly and eventually the plot became available.

TC-Hotel-1 TC-Hotel-2

Version

2.4.3-rc1

What platform are you using?

Linux

What ui mode are you using?

GUI

Relevant log output

2024-08-06T21:16:39.273 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 6cb08fa5af3197a2ec28c4d7ab054c7fc6e10f6cd5aad28f74136419c563f104 new-peak-prev: d611bc47b5ab80180d541c5ede366d82c547ff8a44874ab2b70a3cdfa8c44aec coins: not set
2024-08-06T21:18:51.352 wallet chia.wallet.wallet_node    : ERROR    Error syncing to PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444)
Traceback (most recent call last):
  File "chia/wallet/wallet_node.py", line 1217, in new_peak_from_untrusted
  File "chia/wallet/wallet_node.py", line 1226, in long_sync_from_untrusted
  File "chia/wallet/wallet_node.py", line 1355, in fetch_and_update_weight_proof
Exception: weight proof response was none
2024-08-06T21:18:51.353 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.353 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.353 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.354 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.354 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.354 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.354 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.354 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.355 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.355 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.355 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.355 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.355 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.356 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.356 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.356 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.356 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.356 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.357 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.357 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.357 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.357 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.357 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.357 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.358 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.358 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.358 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time.
2024-08-06T21:18:51.358 wallet wallet_server              : WARNING  Banning 95.175.123.111 for 120 seconds
2024-08-06T21:18:51.358 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.358 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:18:51.358 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.
2024-08-06T21:18:51.358 wallet wallet_server              : WARNING  Banning 35.93.231.21 for 120 seconds
2024-08-06T21:23:16.282 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 6cb08fa5af3197a2ec28c4d7ab054c7fc6e10f6cd5aad28f74136419c563f104 new-peak-prev: d611bc47b5ab80180d541c5ede366d82c547ff8a44874ab2b70a3cdfa8c44aec coins: not set
2024-08-06T21:25:34.365 full_node full_node_server        : WARNING  Connection failed to wss://136.37.132.123:58444/ws. Error: 403, message='Invalid response status', url=URL('wss://136.37.132.123:58444/ws')
2024-08-06T21:26:39.733 full_node chia.full_node.full_node: ERROR    failed fetching 1154934 to 1154965 from peers
2024-08-06T21:26:39.738 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: d0c0f47547a89dab8850de17a4f238be76e76df9513338af44d719768b6bd0f8 new-peak-prev: 88307079501d4d84ab8cd1add5a7e3e2f042773d7026cecc7d1a1b4ce56845db coins: set
2024-08-06T21:28:43.580 full_node full_node_server        : WARNING  Banning 183.245.13.9 for 10 seconds
2024-08-06T21:28:43.581 full_node full_node_server        : WARNING  Invalid handshake with peer PeerInfo(_ip=IPv4Address('183.245.13.9'), _port=18444). Maybe the peer is running old software.
2024-08-06T21:30:33.266 full_node chia.full_node.full_node: ERROR    failed fetching 1155797 to 1155828 from peers
2024-08-06T21:30:33.279 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 40ee5c8b32554a038fbb471d41fd39abe5efee05f38c14f6464348b2006f95b0 new-peak-prev: f997ca4d506febf6984927f2997554d631442ce30439091d2408d021fb530c6c coins: set
2024-08-06T21:33:57.878 full_node chia.full_node.full_node: ERROR    failed fetching 1155828 to 1155859 from peers
2024-08-06T21:33:57.885 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 1ac2c881b2bf6fd1d599b82769d77b08df0323bd1000d063137a011d0cccb2cf new-peak-prev: 4fc28805b4412b125ba64444eb875a1346107c0fc61dd08a9bb1b040c7131f47 coins: set
2024-08-06T21:37:41.926 full_node chia.full_node.full_node: ERROR    failed fetching 1156659 to 1156690 from peers
2024-08-06T21:37:41.929 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: bedb7b73e6475507c772b66e0e0de1a6edcc32c9e5a54bd90d3c540e6605f20a new-peak-prev: ede79083918d1e884b220e2b61e263619fb19e94291de2d7c3ed926dd7993a6e coins: set
2024-08-06T21:39:33.594 full_node chia.full_node.full_node: ERROR    failed fetching 1156850 to 1156881 from peers
2024-08-06T21:39:33.598 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 0f6afb6c1b04cf99ec0245c3e327ce221881033eff2f49a7fddb78de19efeafd new-peak-prev: df25c235946ea6bfc3711a60c9d77eba4d93ee029108e2c45c4655b76520d049 coins: set
2024-08-06T21:40:57.919 full_node chia.full_node.full_node: ERROR    failed fetching 1156913 to 1156944 from peers
2024-08-06T21:40:57.922 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 2130845e6de7fad2f1524ef6975ec0509ea2f84eef314e28a97ac10d0b6fcb65 new-peak-prev: f217f356a5b99e685b03c6a32c4857479c729bd1079b1cb3c858772c70316b33 coins: set
2024-08-06T21:42:54.613 full_node full_node_server        : WARNING  Banning 65.21.193.9 for 10 seconds
2024-08-06T21:42:54.614 full_node full_node_server        : WARNING  Invalid handshake with peer PeerInfo(_ip=IPv4Address('65.21.193.9'), _port=58444). Maybe the peer is running old software.

chia-rschmidt avatar Aug 07 '24 03:08 chia-rschmidt

This is odd, would be nice to have INFO logs

2024-08-06T21:18:51.354 wallet wallet_server : WARNING Banning 95.175.123.111 for 120 seconds 2024-08-06T21:18:51.354 wallet chia.wallet.wallet_node : WARNING Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time. 2024-08-06T21:18:51.354 wallet wallet_server : WARNING Banning 35.93.231.21 for 120 seconds 2024-08-06T21:18:51.354 wallet chia.wallet.wallet_node : WARNING Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time. 2024-08-06T21:18:51.354 wallet wallet_server : WARNING Banning 35.93.231.21 for 120 seconds 2024-08-06T21:18:51.355 wallet chia.wallet.wallet_node : WARNING Peer PeerInfo(_ip=IPv4Address('95.175.123.111'), _port=58444) did not respond in time. 2024-08-06T21:18:51.355 wallet wallet_server : WARNING Banning 95.175.123.111 for 120 seconds 2024-08-06T21:18:51.355 wallet chia.wallet.wallet_node : WARNING Peer PeerInfo(_ip=IPv4Address('35.93.231.21'), _port=58444) did not respond in time.

wjblanke avatar Aug 14 '24 16:08 wjblanke

Hey @chia-rschmidt , can you reattempt the plot creation on rc3 with the logs set to INFO?

  • stop chia
  • run chia configure --log-level INFO
  • start chia and reattempt the k32 creation

BrandtH22 avatar Aug 14 '24 17:08 BrandtH22

debug.log

I was able to recreate on RC3. I also started getting warnings about No space left on device when I had over 300 gb available before plotting. Looking at the folders the temp plotting folder was at 320.5 gb when the plot creation said I would only need 239 gb of temporary space. This was a Chia Proof of Space 2.0.4 plot with K=32.

chia-rschmidt avatar Aug 15 '24 14:08 chia-rschmidt

Hey @chia-rschmidt , two questions to help troubleshoot further:

  • are you using the same drive for the temp, final, and the blockchain db?
  • did you clear the contents of the temp location prior to reattempting?

Lastly take note that the plot sizes are listed in GiB and not GB so there is a slight sizing difference between the two units being discussed. Likely not the issue but I wanted to make sure we are talking apples

BrandtH22 avatar Aug 19 '24 17:08 BrandtH22

This issue has not been updated in 14 days and is now flagged as stale. If this issue is still affecting you and in need of further review, please comment on it with an update to keep it from auto closing in 7 days.

github-actions[bot] avatar Sep 03 '24 11:09 github-actions[bot]

This issue was automatically closed because it has been flagged as stale, and subsequently passed 7 days with no further activity from the submitter or watchers.

github-actions[bot] avatar Sep 11 '24 11:09 github-actions[bot]