decred
DCR native wallet sync stuck at 99.9%
Likely duplicate of https://github.com/decred/dcrdex/issues/2857, https://github.com/decred/dcrdex/issues/2917
I've been running Bison Wallet 1.0.0 for several hours now and was gonna place a DCR/USDT trade but got an error dcr still syncing. progress = 99.90%:
I have mostly decent network connection (and the rest of Bison Wallet functionality works fine), trading overall works fine so far. Not using any privacy or staking functionality of DCR native wallet either.
Restarting Bison Wallet helped immediately, no idea how to reproduce.
My guess would be:
- perhaps my internet connection got dropped at some point and DCR native wallet couldn't properly recover from it once it got back up
- or there is some kind of dead lock
@norwnd Have you tried restarting the Bison Wallet or checking your internet connection to see if that helps?
This is mentioned in issue description.
Possibly related to #2971, as that bug happens at the end of sync.
If you still have the logs, do you see anything (ERR) about a channel being blocked? Or are there logs like in #2857 ?
Possibly related to https://github.com/decred/dcrdex/issues/2971, as that bug happens at the end of sync.
If you still have the logs, do you see anything (ERR) about a channel being blocked? Or are there logs like in https://github.com/decred/dcrdex/issues/2857 ?
Yes I still have the logs, going over those I don't see any "suspicious" messages you posted in https://github.com/decred/dcrdex/issues/2971 or https://github.com/decred/dcrdex/issues/2857, in fact I don't see any errors related to DCR wallet except for this sequence of events:
2024-09-28 17:18:36.089 [DBG] CORE: Wallet synced for asset dcr
...
2024-09-29 10:03:44.741 [WRN] CORE: Wallet for asset dcr has zero network peers!
2024-09-29 10:03:44.748 [WRN] CORE: notify: |WARNING| (walletconfig) Wallet network issue - Decred wallet has no network peers!
...
2024-09-29 21:29:55.452 [ERR] WEB: error placing order: dcr still syncing. progress = 99.90%
...
That suggests to me: originally DCR wallet was successfully synced to 100% and then it lost peer connections (presumably due to my laptop sleeping or network dropping or both), and then nothing really happened as per my logs - I just couldn't place a DCR-related trade - it didn't recover from this until I restarted Bison wallet (even though in UI, as you can see on my screenshot, the Peer Count for DCR wallet always showed as 8 and I was able to place Polygon USDC/USDT trades just fine at all times).
Feels like 1 or 2 of these isn't working correctly for DCR SPV wallet:
- handling peer-rediscovery(reconnect) after different kinds of failures (network outage / laptop sleep)
- displaying correct number of active peer connections in UI
I don't have neutrino logs for this time period though (looks like only couple of days of those are stored by default).
Hey @JoeGruffins, I've encountered this again today,
- like previously, there is nothing interesting in dexc logs related to this issue (as far as I can tell - if you want to check for yourself message me in Element I'll send these over to you)
- in dcrwallet neutrino logs I also don't see any errors (or anything that would fully pin this issue down) - although there is quite a bit of spam related to inability to connect to peer(s) - here is the relevant part you might want to inspect - but what's interesting is seems like dcrwallet keeps receiving/connecting new blocks while dexc still shows 99.9% sync
- at some point 99.9% dropped to 99.4%, and got back up to 99.9% again
- in case it helps, I've also tried sending some DCR from my other wallet into dexc one (while sync is at 99.9%) and it didn't change anything for dexc DCR wallet (sync is still at 99.9% and incoming transaction didn't show up) even though in dcrwallet logs it did show up (with
Inserting unconfirmed transaction ...)
Btw, after I killed Bison wallet process I've got 8 log lines with SPV: Lost peer in neutrino logs (which seems to additionally confirm that I was connected to 8 peers after all).
Update:
I was able to easily reproduce this by unplugging my internet cable and plugging it back in after 15 minutes.
Plus I've also have a go-routine dump if you want to inspect that (I've been running Bison wallet with pprof).
@norwnd thanks
#3005 fixes a bug that can happen at the end of sync that potentially blocks three separate mutexes. I'm hopefully optimistic that those changes also fix this bug. Please let me know if you encounter it on a build with those changes, that would be very helpful. Thanks again.
Cool, thank you!
Please let me know if you encounter it on a build with those changes, that would be very helpful.
Looking forward to the next Bison Wallet release, I'll post an update here if it fixes this issues or not.
It's pretty simple to reproduce too - by unplugging my internet cable and plugging it back in after 15 minutes - if you want to test for yourself (I just don't have my dev environment set up anymore to test against master).
Looking forward to the next Bison Wallet release, I'll post an update here if it fixes this issues or not.
Hey @JoeGruffins, the issue is still there with 1.0.2 release (easily reproducible by unplugging internet for ~30 minutes, and plugging it back in after) - DCR block sync stuck at 99.9$%
Update: don't even need to wait 30 minutes, just reproduced it with 1 minute network outage - this makes it very nasty issue because when it happens it pretty much stops from working all trading activity related to DCR.
I also just saw this on mainnet with the tip as atomic commit in, so that doesn't fix it. Really need to figure this one out
This is at least one part of the problem https://github.com/decred/dcrwallet/issues/2464