dash coinbase transaction (de)serialization issue / wrong txid

[DeckerSU]

There is a bug in mm2 related to DASH coinbase (or not only?) serialization / deserialization. mm2 shows txid of tx 70ce3e63579742f3030e2a263ced5aa7ce606cbacfabdad5718387efac81211a belonged to address XnsqxQLGQv1FTwXJhakqMYL8MpT1G2tegZ as 6899ce7eff3c19ae827d82c575c2486fc0c6102e274cfe83d94604857587d30a. I did a small investigation and here is the results:

1. mm2 output on my_tx_history for address XnsqxQLGQv1FTwXJhakqMYL8MpT1G2tegZ is:

"result": {
    "transactions": [
      {
        "tx_hex": "03000500010000000000000000000000000000000000000000000000000000000000000000ffffffff5803dca01d04018f
0c650cfabe6d6d0000000000000000000000000000000000000000000000000000000000000000010000000000000067f8f92f7f4d62000000000
0156233626333643837323136306162316236303731320000000002ef67c105000000001976a914e976ec3a46c7c8b65e7914e4257c966265b9e7
2488ac1c7d0308000000001976a91485b630193f8d9025495f51b4a3f884d32b92fa1d88ac00000000460200dca01d00d54c0324c6bbfa66bc500
419a0c69249e70ee476f3ea589dad71616ec187d32dc7955b8b2ef67a6b5eaba5343b7770452553be3d9bfd71e7c67d020171d72655",
        "tx_hash": "6899ce7eff3c19ae827d82c575c2486fc0c6102e274cfe83d94604857587d30a",
        "from": [],
        "to": [
          "XnsqxQLGQv1FTwXJhakqMYL8MpT1G2tegZ",
          "XwyHjrAVXFSs2uHKw5BB2U31m8D41izE24"
        ],
        "total_amount": "0",
        "spent_by_me": "0",
        "received_by_me": "1.34446364",
        "my_balance_change": "1.34446364",
        "block_height": 1941724,
        "timestamp": 1695321857,
        "fee_details": {
          "type": "Utxo",
          "coin": "DASH",
          "amount": "0.000000000000000"
        },
        "coin": "DASH",
        "internal_id": "6899ce7eff3c19ae827d82c575c2486fc0c6102e274cfe83d94604857587d30a",
        "transaction_type": "StandardTransfer",
        "memo": null,
        "confirmations": 1688
      },

As we can see tx_hash here is 6899ce7eff3c19ae827d82c575c2486fc0c6102e274cfe83d94604857587d30a, but should be 70ce3e63579742f3030e2a263ced5aa7ce606cbacfabdad5718387efac81211a instead. 2. getrawtransaction for 70ce3e63579742f3030e2a263ced5aa7ce606cbacfabdad5718387efac81211a got directly from dashd is

03000500010000000000000000000000000000000000000000000000000000000000000000ffffffff5803dca01d04018f0c650cfabe6d6d0000000000000000000000000000000000000000000000000000000000000000010000000000000067f8f92f7f4d620000000000156233626333643837323136306162316236303731320000000002ef67c105000000001976a914e976ec3a46c7c8b65e7914e4257c966265b9e72488ac1c7d0308000000001976a91485b630193f8d9025495f51b4a3f884d32b92fa1d88ac00000000460200dca01d00d54c0324c6bbfa66bc500419a0c69249e70ee476f3ea589dad71616ec187d32dc7955b8b2ef67a6b5eaba5343b7770452553be3d9bfd71e7c67d020171d72655

So, from the first look it corresponds tx_hex from mm2 output. But. 3. Let's calculate txid of this tx manually, for example, let's take sha256d hash of raw tx via simple python script:

import hashlib
import binascii
 
def doubleSha256(hex): 
   bin = binascii.unhexlify(hex)
   hash = hashlib.sha256(bin).digest()
   hash2 = hashlib.sha256(hash).digest()
   return binascii.hexlify(hash2)

# Getting raw transaction data
raw = '03000500010000000000000000000000000000000000000000000000000000000000000000ffffffff5803dca01d04018f0c650cfabe6d6d0000000000000000000000000000000000000000000000000000000000000000010000000000000067f8f92f7f4d620000000000156233626333643837323136306162316236303731320000000002ef67c105000000001976a914e976ec3a46c7c8b65e7914e4257c966265b9e72488ac1c7d0308000000001976a91485b630193f8d9025495f51b4a3f884d32b92fa1d88ac00000000460200dca01d00d54c0324c6bbfa66bc500419a0c69249e70ee476f3ea589dad71616ec187d32dc7955b8b2ef67a6b5eaba5343b7770452553be3d9bfd71e7c67d020171d72655'

# Calculating Double-SHA256 hash
hash = doubleSha256(raw)

# Converting result to big-endian hex notation
txid = binascii.hexlify(binascii.unhexlify(hash)[::-1])
txid = str(txid,"ascii")

print("Tx_ID:\n   "+txid)

It will give us 70ce3e63579742f3030e2a263ced5aa7ce606cbacfabdad5718387efac81211a, but mm2 returns tx_hash = 6899ce7eff3c19ae827d82c575c2486fc0c6102e274cfe83d94604857587d30a for this raw tx. 4. Electrum server returns correct data for this tx:

{ "id": 1, "method":"blockchain.scripthash.listunspent", "params":["b85136e199d2b5b0bcfa78c7c293e6c01d76581e0f31f402904754da5a1a700f"] }

Response is:

{"jsonrpc":"2.0","result":[{"tx_hash":"d829ccc4741b9ccc0af4f4d8af66ca6064edff09d7891b351a855cf01714df7e","tx_pos":1,"height":1934382,"value":134438317},{"tx_hash":"75424b78efaa61ce8f0e485bcca80014f3ba20b2f66afec97746264f75a686d7","tx_pos":1,"height":1939138,"value":76054298},{"tx_hash":"70ce3e63579742f3030e2a263ced5aa7ce606cbacfabdad5718387efac81211a","tx_pos":1,"height":1941724,"value":134446364}],"id":1}

blockchain.scripthash.get_history electrum request also gives correct txid, so it's not en Electrum issue. 5. After small debug session i found the error reason. The actial error is in tx_details_by_hash: https://github.com/KomodoPlatform/komodo-defi-framework/blob/79f620559a635704959d3e88067f7ffe5ee30f1f/mm2src/coins/utxo/utxo_common.rs#L3336 Here we get correct raw tx from Electrum or native coin daemon. But later when we calculate tx_hash we did it from tx (pub struct Transaction): https://github.com/KomodoPlatform/komodo-defi-framework/blob/79f620559a635704959d3e88067f7ffe5ee30f1f/mm2src/coins/utxo/utxo_common.rs#L3461-L3462 So, we got tx as deserialize of verbose_tx: https://github.com/KomodoPlatform/komodo-defi-framework/blob/79f620559a635704959d3e88067f7ffe5ee30f1f/mm2src/coins/utxo/utxo_common.rs#L3337 And then inside tx.hash() we hashing serialized tx: https://github.com/KomodoPlatform/komodo-defi-framework/blob/79f620559a635704959d3e88067f7ffe5ee30f1f/mm2src/mm2_bitcoin/chain/src/transaction.rs#L260-L267 So, the tx_hash got like dhash256(serialize(deserialize(verbose_tx))). 6. The main issue that serialize(deserialize(verbose_tx)) for the given tx gives us:

03000500010000000000000000000000000000000000000000000000000000000000000000ffffffff5803dca01d04018f0c650cfabe6d6d0000000000000000000000000000000000000000000000000000000000000000010000000000000067f8f92f7f4d620000000000156233626333643837323136306162316236303731320000000002ef67c105000000001976a914e976ec3a46c7c8b65e7914e4257c966265b9e72488ac1c7d0308000000001976a91485b630193f8d9025495f51b4a3f884d32b92fa1d88ac00000000460200dc

And the real raw tx is longer:

03000500010000000000000000000000000000000000000000000000000000000000000000ffffffff5803dca01d04018f0c650cfabe6d6d0000000000000000000000000000000000000000000000000000000000000000010000000000000067f8f92f7f4d620000000000156233626333643837323136306162316236303731320000000002ef67c105000000001976a914e976ec3a46c7c8b65e7914e4257c966265b9e72488ac1c7d0308000000001976a91485b630193f8d9025495f51b4a3f884d32b92fa1d88ac00000000460200dca01d00d54c0324c6bbfa66bc500419a0c69249e70ee476f3ea589dad71616ec187d32dc7955b8b2ef67a6b5eaba5343b7770452553be3d9bfd71e7c67d020171d72655

7. This is because mm2 doesn't know anything about DASH "tx tail", updated transaction structure introduced in DASH 0.13.0. As we can see we have additional fields here:

"extraPayloadSize": <variable int>,
  "extraPayload": …

For our tx it is:

"extraPayloadSize": 70,
  "extraPayload": "0200dca01d00d54c0324c6bbfa66bc500419a0c69249e70ee476f3ea589dad71616ec187d32dc7955b8b2ef67a6b5eaba5343b7770452553be3d9bfd71e7c67d020171d72655"

8. In the current situation, we have at least two approaches to solve this problem. The first and easiest approach is to use the patch provided in this link: https://gist.github.com/DeckerSU/4f685c0bd21e63c85de8faec39ce56f8. Since all of this code is inside the tx_details_by_hash function, which takes hash: &[u8] as an input argument, there is no need for additional serialization and deserialization of verbose_tx to obtain its hash. We can simply assign tx_hash to the given hash. This approach will always return the correct transaction ID for all cases in which mm2 may incorrectly serialize or deserialize a transaction, such as when there are additional transaction fields that we are not yet aware of.

Another solution is to modify the serialization and deserialization of DASH transactions to include the transaction tail, which consists of everything after nLockTime, such as extraPayloadSize and extraPayload. However, this solution will only fix the "wrong txid" issue for DASH transactions and not for other coins that may also have additional fields in their transactions.

[DeckerSU]

Also, we can check if serialize(deserialize(verbose_tx)) != verbose_tx here and throw an error (put the error in the log) to identify all the such cases.

[cipig]

will this https://github.com/spesmilo/electrumx/pull/234 change something?

[pshenmic]

Dash 0.13 is a fairly old version released in the 2019, there was another CB TX v2 introduced I believe, and there one more ongoing in the Dash v20

Although that could be easier to just pass tx hash, I feel like the proper serialization / deserialization of each known coin will be better in the long run

I eager to provide any necessary help to achieve that regarding Dash protocol changes

[shamardy]

I feel like the proper serialization / deserialization of each known coin will be better in the long run

Thank you for your input @pshenmic, we try to support serialization / deserialization for each known coin but new tx formats sometimes lead to problems. We aim to do this for dash too.

I eager to provide any necessary help to achieve that regarding Dash protocol changes

If you can specify what new fields were added and what fields were removed with each tx version, that would be great help. You can also just point us to these changes in code or any other useful resources and we will pick it up from there :)