dcrwallet Buying tickets fails after reorg in chain ("... references outputs of unknown or fully-spent transaction")

Observed in testnet3. I'm running separate staking (buying tickets) and voting wallets (voting on tickets). I've observed now multiple times that after some chain reorganization, buying tickets fails permanently. Check the attached log file fragment (contains the point in time from where the wallet fails buying tickets). dcrwallet.log

Wallet is happily buying tickets before this point in time.

Then, some reorg happens: 2020-05-30 15:38:05.956 [INF] SYNC: Observed sidechain or orphan block ... 2020-05-30 15:38:28.084 [INF] SYNC: Observed sidechain or orphan block ...

After that, every attempt to buy tickets fails in a similar way: 2020-05-30 15:41:41.522 [INF] SYNC: Connected block 0000000c... 2020-05-30 15:41:41.581 [INF] WLLT: Returning address ... 2020-05-30 15:41:41.582 [INF] WLLT: Returning address ... 2020-05-30 15:41:41.610 [INF] WLLT: Inserting unconfirmed transaction ... 2020-05-30 15:41:41.765 [ERR] WLLT: Abandoning transaction ... which failed to publish 2020-05-30 15:41:41.870 [ERR] TKBY: Ticket purchasing failed: wallet.PurchaseTicketsWithResponse:: dcrd.PublishTransactions:: dcrd.PublishTransaction: rejected transaction 26713da70078ccaccf11f04059523a7944f322c5e3fb049939127efad5aee4dc: orphan transaction 26713da70078ccaccf11f04059523a7944f322c5e3fb049939127efad5aee4dc references outputs of unknown or fully-spent transaction dda821aabe6b2df1a0ff07c2c70a2e11eca598979d6034eb147f2f57d5a45b92

The only way to recover is to completely recreate the walled from seed.

stop/start of dcrwallet does not help
rescanwallet does not help

Jun 03 '20 17:06 hb9xar

Are there any previous references to spending the spent outputs of dda821aabe6b2df1a0ff07c2c70a2e11eca598979d6034eb147f2f57d5a45b92 earlier in the log, or is this the first occurrance? (You might as well just grep for that tx and include all of those messages)

Jun 03 '20 17:06 jrick

Ok, I see something suspicious. These five transactions were removed as the wallet believed they were never mined:

75393009410df19da71bc0e98f414ea8717bdde2f4f419360ee815e96ffc7f03
da5c65e63c25edf445370e4bf3f5af51326d2eb52efcc9365e58b03981be6d4c
fd42c04b4ae105a969bdefd1d5369c1db173163e445172c8f6f17e1d34808752
0f3aa643229765fe2b1f49b23208db04afda49ecfe7ea580863db93552fb4d61
53b3d0fdd00c687a412c57b591ec58dfbdffffce42c441ddb8c78e175a6f6a73

But they are mined. e.g. https://testnet.dcrdata.org/tx/75393009410df19da71bc0e98f414ea8717bdde2f4f419360ee815e96ffc7f03.

What version of the wallet are you running, the latest release or a development build from master? And can you verify with validateaddress that this wallet is tracking the addresses TsfakXkFtqc1UCHbVypaSzJPMxNi2DgFq9x and TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML? (It should report "ismine": true)

Jun 03 '20 18:06 jrick

Hi jrick,

ok, I try try to provide answers:

dcrwallet version dcrwallet version 1.5.2-pre (Go version go1.14.2 linux/amd64)

dcrwallet ist built from GIT, currently using $ git log -n 1 commit f1aa4c354e05aa331bc3148dc0af16e607042e54

I've had the same issue also with earlier versions of dcrwallet (can't provide detailed version info about them, though)

occurences of tx dda821aa... in logs $ grep dda821aabe6b2df1a0ff07c2c70a2e11eca598979d6034eb147f2f57d5a45b92 .dcrwallet-stake/logs/testnet3/dcrwallet.log dcrwallet2.log

verification of addresses TsfakXkFtqc1UCHbVypaSzJPMxNi2DgFq9x and TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML

staking wallet: [decred-test@cryptocurrency dcrwallet]$ dcrctl-stake validateaddress TsfakXkFtqc1UCHbVypaSzJPMxNi2DgFq9x { "isvalid": true, "address": "TsfakXkFtqc1UCHbVypaSzJPMxNi2DgFq9x", "ismine": true, "pubkeyaddr": "TkQ4cGio6Ff7EXvGX1qFPChf5BMgQbSiHrEaiSXqsRx7kx5imNZkj", "pubkey": "03a3a1220d6b44a24b3bca56b8c8530782e05b790c27c14caeaecd265f374450e1", "iscompressed": true, "account": "default" } $ dcrctl-stake validateaddress TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML { "isvalid": true, "address": "TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML" }

The address TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML is owned by my voting wallet, the staking wallet has uses the config: ticketbuyer.votingaddress=TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML

$ dcrctl-vote validateaddress TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML { "isvalid": true, "address": "TsTkgBEUdzTgzwWJHvQdv1vH2LhUZoMzHML", "ismine": true, "pubkeyaddr": "TkKnGvUPwTi8RMoEr4U88YkwDkNoaG7PjbemG7kf91CXzq35gyaYG", "pubkey": "02e31032833e8ad19f0fe53120af2047be216ce797cdbdb6827d1443fe48efc286", "iscompressed": true, "account": "default" }

Note: dcrctl-stake / dcrctl-vote are aliases, poiting to the staking, resp. voting wallet: alias dcrctl-stake='dcrctl --wallet -s 127.0.0.1:29110' alias dcrctl-vote='dcrctl --wallet -s 127.0.0.1:29113'

Jun 03 '20 20:06 hb9xar