rust-payjoin icon indicating copy to clipboard operation
rust-payjoin copied to clipboard
verified publisher icon payjoin

Test compatibility with joinmarket

Open DanGould opened this issue 2 years ago • 14 comments

  • [x] send from JoinMarket to payjoin-cli. JoinMarket may need to be able to trust self-signed certs or otherwise a cert from e.g. letsencrypt may need to be obtained. This is a priority to smoke test compatibility for production integration @ Bull Bitcoin
  • [ ] receive tested from JoinMarket. JoinMarket receives payjoin via Tor, so SOCKS Tor support would need to be added to payjoin-cli

https://github.com/payjoin/payjoin.github.io/issues/8

DanGould avatar Apr 06 '23 19:04 DanGould

I've tested sending from JoinMarket to payjoin-cli and found the bug described below.

Setup notes

JoinMarket doesn't trust self-signed certificates, so I had to obtain a legit certificate from letsencrypt.org as you described. I launched an EC2 instance with nginx acting as a proxy for payjoin-cli, running on signet. I was able to test this setup by sending a successful payjoin transaction from payjoin-cli on my local machine, to the payjoin receiver listening behind the nginx proxy.

Receive error when sending from JM

However, sending from joinmarket using the same setup results in a FinalizeAndExtractPSBT error on the receiver side.

RUST_LOG=debug cargo run -- receive 10000
    Finished dev [unoptimized + debuginfo] target(s) in 0.14s
     Running `/home/ec2-user/rust-payjoin/target/debug/payjoin-cli receive 10000`
[2024-04-18T22:43:46Z DEBUG payjoin_cli::app::config] App config: AppConfig { bitcoind_rpchost: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(38332), path: "/wallet/pj_wallet", query: None, fragment: None }, bitcoind_cookie: None, bitcoind_rpcuser: "***", bitcoind_rpcpass: "***", pj_host: 0.0.0.0:3000, pj_endpoint: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(3000), path: "/", query: None, fragment: None }, sub_only: false }
[2024-04-18T22:43:46Z DEBUG bitcoincore_rpc] JSON-RPC request: getnewaddress [null,null]
Listening at 0.0.0.0:3000. Configured to accept payjoin at BIP 21 Payjoin Uri:
bitcoin:tb1qn9dt8hmcg87cvz2rpmclskvp4lv5xdfj44qh5q?amount=0.0001&pj=https://localhost:3000/&pjos=0
[2024-04-18T22:45:54Z DEBUG payjoin_cli::app::v1] Received request: Request { method: POST, uri: /?v=1&disableoutputsubstitution=true&additionalfeeoutputindex=0&maxadditionalfeecontribution=95&minfeerate=1.1, version: HTTP/1.0, headers: {"host": "pj.spacebear.dev", "x-forwarded-proto": "https", "connection": "close", "content-length": "360", "content-type": "text/plain"}, body: Body(Streaming) }
[2024-04-18T22:45:54Z DEBUG payjoin::receive] Received original psbt: PartiallySignedTransaction { unsigned_tx: Transaction { version: 2, lock_time: Blocks(Height(191867)), input: [TxIn { previous_output: OutPoint { txid: 0xc6ffb1bb73040f88b520cd7869e28b2cbeebc320cc42f67a84e3107306a9ea3b, vout: 1 }, script_sig: Script(), sequence: Sequence(4294967293), witness: Witness { content: [], witness_elements: 0, indices_start: 0 } }], output: [TxOut { value: 949217, script_pubkey: Script(OP_0 OP_PUSHBYTES_20 35bef6d37390ea20c43ba951cc2a30831166399b) }, TxOut { value: 10000, script_pubkey: Script(OP_0 OP_PUSHBYTES_20 995ab3df7841fd8609430ef1f85981afd9433532) }] }, version: 0, xpub: {}, proprietary: {}, unknown: {}, inputs: [Input { non_witness_utxo: None, witness_utxo: Some(TxOut { value: 959382, script_pubkey: Script(OP_0 OP_PUSHBYTES_20 4d20ca992bee15e856dfefa1be0b1ad2a0c69fc9) }), partial_sigs: {}, sighash_type: None, redeem_script: None, witness_script: None, bip32_derivation: {}, final_script_sig: None, final_script_witness: Some(Witness { content: [71, 48, 68, 2, 32, 110, 221, 75, 43, 61, 10, 149, 172, 84, 232, 107, 84, 126, 26, 61, 206, 44, 218, 21, 171, 210, 9, 24, 3, 3, 153, 13, 99, 28, 206, 92, 30, 2, 32, 35, 237, 171, 254, 193, 89, 175, 223, 188, 120, 4, 209, 22, 202, 10, 145, 197, 227, 255, 25, 70, 229, 76, 198, 51, 21, 227, 222, 37, 174, 113, 209, 1, 33, 3, 133, 51, 208, 148, 147, 127, 254, 66, 145, 179, 7, 93, 245, 123, 11, 154, 188, 30, 55, 35, 40, 97, 250, 126, 181, 20, 231, 187, 3, 108, 118, 3, 0, 0, 0, 0, 72, 0, 0, 0], witness_elements: 2, indices_start: 106 }), ripemd160_preimages: {}, sha256_preimages: {}, hash160_preimages: {}, hash256_preimages: {}, tap_key_sig: None, tap_script_sigs: {}, tap_scripts: {}, tap_key_origins: {}, tap_internal_key: None, tap_merkle_root: None, proprietary: {}, unknown: {} }], outputs: [Output { redeem_script: None, witness_script: None, bip32_derivation: {}, tap_internal_key: None, tap_tree: None, tap_key_origins: {}, proprietary: {}, unknown: {} }, Output { redeem_script: None, witness_script: None, bip32_derivation: {}, tap_internal_key: None, tap_tree: None, tap_key_origins: {}, proprietary: {}, unknown: {} }] }
[2024-04-18T22:45:54Z DEBUG payjoin::receive::optional_parameters] parsed optional parameters: Params { disable_output_substitution: true, additional_fee_contribution: Some((Amount(0.00000095 BTC), 0)), min_feerate: FeeRate(275) }
[2024-04-18T22:45:54Z DEBUG payjoin::receive] Received request with params: Params { disable_output_substitution: true, additional_fee_contribution: Some((Amount(0.00000095 BTC), 0)), min_feerate: FeeRate(275) }
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getblockchaininfo []
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getnetworkinfo []
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: testmempoolaccept [["020000000001013beaa9067310e3847af642cc20c3ebbe2c8be26978cd20b5880f0473bbb1ffc60100000000fdffffff02e17b0e000000000016001435bef6d37390ea20c43ba951cc2a30831166399b1027000000000000160014995ab3df7841fd8609430ef1f85981afd94335320247304402206edd4b2b3d0a95ac54e86b547e1a3dce2cda15abd209180303990d631cce5c1e022023edabfec159afdfbc7804d116ca0a91c5e3ff1946e54cc63315e3de25ae71d10121038533d094937ffe4291b3075df57b0b9abc1e37232861fa7eb514e7bb036c76037bed0200"]]
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getaddressinfo ["tb1qf5sv4xftac27s4kla7smuzc662svd87fgs02yu"]
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getaddressinfo ["tb1qxkl0d5mnjr4zp3pm49guc23ssvgkvwvm429wyj"]
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getaddressinfo ["tb1qn9dt8hmcg87cvz2rpmclskvp4lv5xdfj44qh5q"]
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: listunspent []
[2024-04-18T22:45:54Z DEBUG payjoin_cli::app] selected utxo: ListUnspentResultEntry {
        txid: 0x76262ad10fbbc4692239d74e261ebf9ecb19ddeaea10460e38cf0e1ab8f1e0ad,
        vout: 0,
        address: Some(
            Address<NetworkUnchecked>(TB1QKRMXNUPZNEVSNPUVY9VAN6TGA9UHZGQPJ39FNG),
        ),
        label: Some(
            "",
        ),
        redeem_script: None,
        witness_script: None,
        script_pub_key: Script(OP_0 OP_PUSHBYTES_20 b0f669f0229e5909878c2159d9e968e979712001),
        amount: Amount(0.00899792 BTC),
        confirmations: 1,
        spendable: true,
        solvable: true,
        descriptor: Some(
            "wpkh([90a5aa64/84h/1h/0h/0/20]02ac17bc19cf73e28f1eb29343f0a30e333fdac505c0502e4886b990baa6c69ba7)#gljrxr7w",
        ),
        safe: true,
    }
[2024-04-18T22:45:54Z DEBUG bitcoincore_rpc] JSON-RPC request: getnewaddress [null,null]
[2024-04-18T22:45:55Z DEBUG payjoin::receive] min_feerate: FeeRate(275)
[2024-04-18T22:45:55Z DEBUG bitcoincore_rpc] JSON-RPC request: walletprocesspsbt ["cHNidP8BAJoCAAAAAq3g8bgaDs84DkYQ6urdGcuevx4mTtc5ImnEuw/RKiZ2AAAAAAD9////O+qpBnMQ44R69kLMIMPrviyL4ml4zSC1iA8Ec7ux/8YBAAAAAP3///8ClnsOAAAAAAAWABQ1vvbTc5DqIMQ7qVHMKjCDEWY5m+DhDQAAAAAAFgAUWIMWf8n1WEfQh3v8yh+J7jCs/a577QIAAAEBH9C6DQAAAAAAFgAUsPZp8CKeWQmHjCFZ2elo6XlxIAEAAQEflqMOAAAAAAAWABRNIMqZK+4V6Fbf76G+CxrSoMafyQEIawJHMEQCIG7dSys9CpWsVOhrVH4aPc4s2hWr0gkYAwOZDWMczlweAiAj7av+wVmv37x4BNEWygqRxeP/GUblTMYzFePeJa5x0QEhA4Uz0JSTf/5CkbMHXfV7C5q8HjcjKGH6frUU57sDbHYDAAAA",true,"ALL",false]
[2024-04-18T22:45:55Z DEBUG bitcoincore_rpc] JSON-RPC error for walletprocesspsbt: RpcError { code: -1, message: "Internal bug detected: FinalizeAndExtractPSBT(psbtx, mtx)\nwallet/rpc/spend.cpp:1619 (operator())\nBitcoin Core v26.0.0\nPlease report this issue here: https://github.com/bitcoin/bitcoin/issues\n", data: None }
[2024-04-18T22:45:55Z ERROR payjoin_cli::app::v1] Error handling request: Internal Server Error: JSON-RPC error: RPC error response: RpcError { code: -1, message: "Internal bug detected: FinalizeAndExtractPSBT(psbtx, mtx)\nwallet/rpc/spend.cpp:1619 (operator())\nBitcoin Core v26.0.0\nPlease report this issue here: https://github.com/bitcoin/bitcoin/issues\n", data: None }

spacebear21 avatar Apr 18 '24 23:04 spacebear21

Awesome seeing this compatibility put to the test.

This is the decoded psbt that caused the error which may help diagnose the problem

bitcoin-cli decodepsbt "cHNidP8BAJoCAAAAAq3g8bgaDs84DkYQ6urdGcuevx4mTtc5ImnEuw/RKiZ2AAAAAAD9////O+qpBnMQ44R69kLMIMPrviyL4ml4zSC1iA8Ec7ux/8YBAAAAAP3///8ClnsOAAAAAAAWABQ1vvbTc5DqIMQ7qVHMKjCDEWY5m+DhDQAAAAAAFgAUWIMWf8n1WEfQh3v8yh+J7jCs/a577QIAAAEBH9C6DQAAAAAAFgAUsPZp8CKeWQmHjCFZ2elo6XlxIAEAAQEflqMOAAAAAAAWABRNIMqZK+4V6Fbf76G+CxrSoMafyQEIawJHMEQCIG7dSys9CpWsVOhrVH4aPc4s2hWr0gkYAwOZDWMczlweAiAj7av+wVmv37x4BNEWygqRxeP/GUblTMYzFePeJa5x0QEhA4Uz0JSTf/5CkbMHXfV7C5q8HjcjKGH6frUU57sDbHYDAAAA"
{
  "tx": {
    "txid": "d839f0dbd2f27817d0be1920b346001ca110c591b7f7928fc2d349dece95a0d7",
    "hash": "d839f0dbd2f27817d0be1920b346001ca110c591b7f7928fc2d349dece95a0d7",
    "version": 2,
    "size": 154,
    "vsize": 154,
    "weight": 616,
    "locktime": 191867,
    "vin": [
      {
        "txid": "76262ad10fbbc4692239d74e261ebf9ecb19ddeaea10460e38cf0e1ab8f1e0ad",
        "vout": 0,
        "scriptSig": {
          "asm": "",
          "hex": ""
        },
        "sequence": 4294967293
      },
      {
        "txid": "c6ffb1bb73040f88b520cd7869e28b2cbeebc320cc42f67a84e3107306a9ea3b",
        "vout": 1,
        "scriptSig": {
          "asm": "",
          "hex": ""
        },
        "sequence": 4294967293
      }
    ],
    "vout": [
      {
        "value": 0.00949142,
        "n": 0,
        "scriptPubKey": {
          "asm": "0 35bef6d37390ea20c43ba951cc2a30831166399b",
          "desc": "addr(bc1qxkl0d5mnjr4zp3pm49guc23ssvgkvwvmlv7alp)#gjwwrf32",
          "hex": "001435bef6d37390ea20c43ba951cc2a30831166399b",
          "address": "bc1qxkl0d5mnjr4zp3pm49guc23ssvgkvwvmlv7alp",
          "type": "witness_v0_keyhash"
        }
      },
      {
        "value": 0.00909792,
        "n": 1,
        "scriptPubKey": {
          "asm": "0 5883167fc9f55847d0877bfcca1f89ee30acfdae",
          "desc": "addr(bc1qtzp3vl7f74vy05y8007v58ufacc2eldwgkdk6y)#kljvka6k",
          "hex": "00145883167fc9f55847d0877bfcca1f89ee30acfdae",
          "address": "bc1qtzp3vl7f74vy05y8007v58ufacc2eldwgkdk6y",
          "type": "witness_v0_keyhash"
        }
      }
    ]
  },
  "global_xpubs": [
  ],
  "psbt_version": 0,
  "proprietary": [
  ],
  "unknown": {
  },
  "inputs": [
    {
      "witness_utxo": {
        "amount": 0.00899792,
        "scriptPubKey": {
          "asm": "0 b0f669f0229e5909878c2159d9e968e979712001",
          "desc": "addr(bc1qkrmxnupznevsnpuvy9van6tga9uhzgqpch76gm)#z2welvn9",
          "hex": "0014b0f669f0229e5909878c2159d9e968e979712001",
          "address": "bc1qkrmxnupznevsnpuvy9van6tga9uhzgqpch76gm",
          "type": "witness_v0_keyhash"
        }
      }
    },
    {
      "witness_utxo": {
        "amount": 0.00959382,
        "scriptPubKey": {
          "asm": "0 4d20ca992bee15e856dfefa1be0b1ad2a0c69fc9",
          "desc": "addr(bc1qf5sv4xftac27s4kla7smuzc662svd87fzk5el0)#4y3q79p9",
          "hex": "00144d20ca992bee15e856dfefa1be0b1ad2a0c69fc9",
          "address": "bc1qf5sv4xftac27s4kla7smuzc662svd87fzk5el0",
          "type": "witness_v0_keyhash"
        }
      },
      "final_scriptwitness": [
        "304402206edd4b2b3d0a95ac54e86b547e1a3dce2cda15abd209180303990d631cce5c1e022023edabfec159afdfbc7804d116ca0a91c5e3ff1946e54cc63315e3de25ae71d101",
        "038533d094937ffe4291b3075df57b0b9abc1e37232861fa7eb514e7bb036c7603"
      ]
    }
  ],
  "outputs": [
    {
    },
    {
    }
  ],
  "fee": 0.00000240
}

DanGould avatar Apr 19 '24 03:04 DanGould

This is what i think is happening:

  1. Joinmarket sends the PSBT with its own input signed.
    • This PSBT input only includes a witness_utxo (no non_witness_utxo), which is legal/encouraged by BIP 174.
    • The final_scriptwitness for that input has a signature which commits to all outputs in the transaction.
  2. The payjoin-cli receiver runs some checks, then substitutes its own output address (scriptPubKey) with a new one.
    • Here, the receiver appears to disregard the disableoutputsubstitution=true flag in the POST request from JM sender. It substitutes its output regardless.
  3. The payjoin-cli receiver attempts to finalize the psbt via walletprocesspsbt RPC call, but the signature in final_scriptwitness from (1) is no longer valid due to the output substitution.
    • Bitcoin Core fails to FinalizeAndExtractPSBT and throws a CHECK_NONFATAL failure (It should probably handle this scenario gracefull instead...)
    • iiuc the PSBT doesn't need to be finalized at this stage (payjoin proposal). I can get walletprocesspsbt to work on this PSBT manually via bitcoin-cli by setting the finalize flag to false. Unfortunately it looks like the rust bitcoincore-rpc library doesn't support finalize yet.

The reason it works with the payjoin-cli sender is because the sender populates both witness_utxo and non_witness_utxo for the input(s) it contributes, which is also allowed by BIP 174. Bitcoin Core sees the non_witness_utxo, so it doesn't require a valid witness sig to finalize the PSBT.

TL;DR:

  • payjoin-cli receiver should respect disableoutputsubstitution. See: https://github.com/payjoin/rust-payjoin/issues/238
  • payjoin-cli receiver doesn't need to finalize PSBT proposals before sending them back to the sender.

spacebear21 avatar Apr 29 '24 17:04 spacebear21

Given your new understanding of the problem, what sense do you make of these action items? Which ones do you think should be done, and which, if any, ignored so we can focus on the mission to scale bitcoin and better preserve privacy for as much bitcoin volume as possible?

  1. Respect disableoutputsubstitution #238. Are you going to take this one on @grizznaut ?
  2. Open an issue in Core describing the walletprocesspsbt issue as stated here to fail more gracefully in the future.
  3. Update the readme to reflect that JoinMarket has been tested but is incompatible with our current implementation with a basic summary of the reasons you outlined above. Separate payjoin-cli from the payjoin crate for this purpose. Is the problem only because of CLI which would otherwise work with the payjoin crate?
  4. Add finalize parameter to bitcoincore-rpc in the repo

DanGould avatar Apr 30 '24 17:04 DanGould

  1. Respect disableoutputsubstitution payjoin-cli should respect output substitution #238. Are you going to take this one on @grizznaut ?

This seems like the most important point to address. I'll put up a PR for it.

  1. Open an issue in Core describing the walletprocesspsbt issue as stated here to fail more gracefully in the future.

This is low priority but would be nice to get a fix in Core. I'll open an issue over there at some point when I can repro in a test.

  1. Update the readme to reflect that JoinMarket has been tested but is incompatible with our current implementation with a basic summary of the reasons you outlined above. Separate payjoin-cli from the payjoin crate for this purpose. Is the problem only because of CLI which would otherwise work with the payjoin crate?

Yes, I believe the issue is only because of the payjoin-cli implementation. I'll do another test payjoin from joinmarket once I have a fix for (1) to confirm that it works, then update the readme accordingly.

  1. Add finalize parameter to bitcoincore-rpc in the repo

It looks like Core 22.0 and above are not officially supported by bitcoincore-rpc (based on readme), but the finalize param was added in 23.0, so it likely won't get added anytime soon. This is less important though, since it only affects output substitution when there are witness_utxo-only inputs from the sender.

spacebear21 avatar May 01 '24 00:05 spacebear21

The output substitution fix is here: https://github.com/payjoin/rust-payjoin/pull/239.

Unfortunately this isn't enough to fix the joinmarket->payjoin-cli flow because the transaction details still gets updated in other places (receiver input, sender fees) so the sender witness sig is still not valid. I don't see a way around this using Core <23.0...

Updated readme: https://github.com/payjoin/rust-payjoin/pull/240.

spacebear21 avatar May 01 '24 13:05 spacebear21

The payjoin-cli receiver attempts to finalize the psbt via walletprocesspsbt RPC call, but the signature in final_scriptwitness from (1) is no longer valid due to the output substitution.

By the time the receiver is applying its own signatures, it should have extracted the valid Original PSBT Transaction and may remove the final_scriptwitness. ProvisionalProposal::prepare_psbt removes the final_scriptwitness from sender inputs after wallet_process_psbt is called already. Would removing this data before processing the PSBT, perhaps in ProvisionalProposal::finalize_proposal not fix the problem and allow the protocol to proceed?

If not, shall we open an issue on JoinMarket to ask them to accomodate the limitation on bitcoind rpc?

DanGould avatar May 01 '24 21:05 DanGould

Hmm good point, that could work. I'll try to patch it in and attempt another receive from joinmarket.

spacebear21 avatar May 02 '24 20:05 spacebear21

Nice, that worked! Successfully received from joinmarket with these changes.

spacebear21 avatar May 03 '24 01:05 spacebear21

Now the question is: do we test the sender with joinmarket?

I think it makes sense to test and document that a sender works with a custom Tor setup with PDK, To ensure we've got the spec correct as far as JoinMarket is concerned, but I don't think it makes sense to get all of the Tor support production-ready in payjoin CLI.

One might also consider this a lot of work to test whether JoinMarket can receive payjoins from PDK relative to its impact to affect getting integrations deployed everywhere.

@grizznaut shall we close this issue or do you see a relatively short path to hooking up a Tor proxy to a custom PDK integration that you would be interested in testing?

DanGould avatar May 07 '24 20:05 DanGould

I haven't looked into the JoinMarket receiver flow yet. I agree that it would be great to at least try it with PDK on a custom Tor setup and close the loop on these compatibility tests. I'll explore this and see if I can get a working setup within the week, and post updates here.

spacebear21 avatar May 07 '24 21:05 spacebear21

I was able to hook this up by using socat like this:

$ socat TCP-LISTEN:8888 SOCKS4A:localhost:<JM hidden service address>.onion:80,socksport=9150
$ RUST_LOG=trace cargo run -- send "bitcoin:tb1q7qklxtkpprwrnp4k8v5zd2f9ccqcraaaz7pad7?amount=0.0001&pj=http://localhost:8888" --fee-rate 1

This required a small hack to allow http by commenting out this line.

But, the payjoin proposal returned by joinmarket failed the input checks:

[2024-05-08T14:00:36Z DEBUG bitcoincore_rpc] JSON-RPC request: walletcreatefundedpsbt [[],{"tb1q7qklxtkpprwrnp4k8v5zd2f9ccqcraaaz7pad7":0.0001},0,{"feeRate":0.00001,"lockUnspents":true}]
[2024-05-08T14:00:36Z TRACE bitcoincore_rpc] JSON-RPC response for walletcreatefundedpsbt: {"psbt":"cHNidP8BAJoCAAAAAvc36ET8IZrFp6mFLUCxUtYztbhON3v93iSE1XfbGJmXAQAAAAD9////MlEnugogxC+v+ScCjWYcW/DkR1D+mvV8jX/eRXj3q78BAAAAAP3///8CECcAAAAAAAAWABTwLfMuwQjcOYa2OygmqSXGAYH3vXAlAAAAAAAAFgAU/O4qO0p5pzWJfNjaiwrgiRJeeuwAAAAAAAEAmgIAAAACVTl8glG9P1RXliQ5R6QkDA9w3TkrpDOzzxun5SK8fLUBAAAAAP3////8mLneH0S6ZbwKj3VaD44+P0K2bSRAVO9jX2TwOHbUPQAAAAAA/f///wIQJwAAAAAAABYAFAZUyGFoDD5OwWjiQQ9nEcKdaAsFQCYAAAAAAAAWABThi4ZUDn9JXf2IaH2mW2tDy85jlAAAAAABAR9AJgAAAAAAABYAFOGLhlQOf0ld/YhofaZba0PLzmOUIgYDfODc/LsQumEV+sngZrxNhicYnDayfFCB14TJFh957aEYkKWqZFQAAIABAACAAAAAgAEAAAADAAAAAAEAcQIAAAAB5UsgcO9iX00hnAJ4BTybFkQcS2lP5dauhvfBNwrmeDIAAAAAAP3///8ChbUNAAAAAAAWABQozLLATs9ncM0Vz6AnQ7CxxEYJARAnAAAAAAAAFgAU0WIqXjobehPOTVyJ4Nf2uSezBtZA9AIAAQEfECcAAAAAAAAWABTRYipeOht6E85NXIng1/a5J7MG1iIGA87mZKs054C3t7HKDyox5A14XQ+e1dtVv+yrlAXRzjAUGJClqmRUAACAAQAAgAAAAIAAAAAAKAAAAAAAIgICtSx0tOeF9mJJSTbBuza1m0907HVxqr51/uN07sNP95cYkKWqZFQAAIABAACAAAAAgAEAAAAEAAAAAA==","fee":0.00000208,"changepos":1}
[2024-05-08T14:00:36Z DEBUG bitcoincore_rpc] JSON-RPC request: walletprocesspsbt ["cHNidP8BAJoCAAAAAvc36ET8IZrFp6mFLUCxUtYztbhON3v93iSE1XfbGJmXAQAAAAD9////MlEnugogxC+v+ScCjWYcW/DkR1D+mvV8jX/eRXj3q78BAAAAAP3///8CECcAAAAAAAAWABTwLfMuwQjcOYa2OygmqSXGAYH3vXAlAAAAAAAAFgAU/O4qO0p5pzWJfNjaiwrgiRJeeuwAAAAAAAEAmgIAAAACVTl8glG9P1RXliQ5R6QkDA9w3TkrpDOzzxun5SK8fLUBAAAAAP3////8mLneH0S6ZbwKj3VaD44+P0K2bSRAVO9jX2TwOHbUPQAAAAAA/f///wIQJwAAAAAAABYAFAZUyGFoDD5OwWjiQQ9nEcKdaAsFQCYAAAAAAAAWABThi4ZUDn9JXf2IaH2mW2tDy85jlAAAAAABAR9AJgAAAAAAABYAFOGLhlQOf0ld/YhofaZba0PLzmOUIgYDfODc/LsQumEV+sngZrxNhicYnDayfFCB14TJFh957aEYkKWqZFQAAIABAACAAAAAgAEAAAADAAAAAAEAcQIAAAAB5UsgcO9iX00hnAJ4BTybFkQcS2lP5dauhvfBNwrmeDIAAAAAAP3///8ChbUNAAAAAAAWABQozLLATs9ncM0Vz6AnQ7CxxEYJARAnAAAAAAAAFgAU0WIqXjobehPOTVyJ4Nf2uSezBtZA9AIAAQEfECcAAAAAAAAWABTRYipeOht6E85NXIng1/a5J7MG1iIGA87mZKs054C3t7HKDyox5A14XQ+e1dtVv+yrlAXRzjAUGJClqmRUAACAAQAAgAAAAIAAAAAAKAAAAAAAIgICtSx0tOeF9mJJSTbBuza1m0907HVxqr51/uN07sNP95cYkKWqZFQAAIABAACAAAAAgAEAAAAEAAAAAA=="]
[2024-05-08T14:00:36Z TRACE bitcoincore_rpc] JSON-RPC response for walletprocesspsbt: {"psbt":"cHNidP8BAJoCAAAAAvc36ET8IZrFp6mFLUCxUtYztbhON3v93iSE1XfbGJmXAQAAAAD9////MlEnugogxC+v+ScCjWYcW/DkR1D+mvV8jX/eRXj3q78BAAAAAP3///8CECcAAAAAAAAWABTwLfMuwQjcOYa2OygmqSXGAYH3vXAlAAAAAAAAFgAU/O4qO0p5pzWJfNjaiwrgiRJeeuwAAAAAAAEAmgIAAAACVTl8glG9P1RXliQ5R6QkDA9w3TkrpDOzzxun5SK8fLUBAAAAAP3////8mLneH0S6ZbwKj3VaD44+P0K2bSRAVO9jX2TwOHbUPQAAAAAA/f///wIQJwAAAAAAABYAFAZUyGFoDD5OwWjiQQ9nEcKdaAsFQCYAAAAAAAAWABThi4ZUDn9JXf2IaH2mW2tDy85jlAAAAAABAR9AJgAAAAAAABYAFOGLhlQOf0ld/YhofaZba0PLzmOUAQhrAkcwRAIgBrX7HIWjcRilydfkAVac6/FLk5jIsjX9t6f62nHY0SICIBn/Gpo457Sk2X7roSROfgf61iOWJ/Cd595YRPSrRMa7ASEDfODc/LsQumEV+sngZrxNhicYnDayfFCB14TJFh957aEAAQBxAgAAAAHlSyBw72JfTSGcAngFPJsWRBxLaU/l1q6G98E3CuZ4MgAAAAAA/f///wKFtQ0AAAAAABYAFCjMssBOz2dwzRXPoCdDsLHERgkBECcAAAAAAAAWABTRYipeOht6E85NXIng1/a5J7MG1kD0AgABAR8QJwAAAAAAABYAFNFiKl46G3oTzk1cieDX9rknswbWAQhrAkcwRAIgENkRiZf3SUjgNbP2ZUgWEeCzzcRYU2t46LoSV0n45zcCIGqS4kgNyzE6skwlflibcO4IumbIXS4t8LD7OzaLy7YWASEDzuZkqzTngLe3scoPKjHkDXhdD57V21W/7KuUBdHOMBQAACICArUsdLTnhfZiSUk2wbs2tZtPdOx1caq+df7jdO7DT/eXGJClqmRUAACAAQAAgAAAAIABAAAABAAAAAA=","complete":true,"hex":"02000000000102f737e844fc219ac5a7a9852d40b152d633b5b84e377bfdde2484d577db1899970100000000fdffffff325127ba0a20c42faff927028d661c5bf0e44750fe9af57c8d7fde4578f7abbf0100000000fdffffff021027000000000000160014f02df32ec108dc3986b63b2826a925c60181f7bd7025000000000000160014fcee2a3b4a79a735897cd8da8b0ae089125e7aec02473044022006b5fb1c85a37118a5c9d7e401569cebf14b9398c8b235fdb7a7fada71d8d122022019ff1a9a38e7b4a4d97eeba1244e7e07fad6239627f09de7de5844f4ab44c6bb0121037ce0dcfcbb10ba6115fac9e066bc4d8627189c36b27c5081d784c9161f79eda102473044022010d9118997f74948e035b3f665481611e0b3cdc458536b78e8ba125749f8e73702206a92e2480dcb313ab24c257e589b70ee08ba66c85d2e2df0b0fb3b368bcbb616012103cee664ab34e780b7b7b1ca0f2a31e40d785d0f9ed5db55bfecab9405d1ce301400000000"}
...
Sending fallback request to http://localhost:8888/?v=1&additionalfeeoutputindex=1&maxadditionalfeecontribution=68&minfeerate=1
[2024-05-08T14:00:36Z DEBUG ureq::stream] connecting to localhost:8888 at [::1]:8888
[2024-05-08T14:00:36Z DEBUG ureq::stream] created stream: Stream(TcpStream { addr: [::1]:60872, peer: [::1]:8888, fd: 12 })
[2024-05-08T14:00:36Z DEBUG ureq::unit] sending request POST http://localhost:8888/?v=1&additionalfeeoutputindex=1&maxadditionalfeecontribution=68&minfeerate=1
[2024-05-08T14:00:36Z DEBUG ureq::unit] writing prelude: POST /?v=1&additionalfeeoutputindex=1&maxadditionalfeecontribution=68&minfeerate=1 HTTP/1.1
    Host: localhost:8888
    User-Agent: ureq/2.9.6
    Accept: */*
    Content-Type: text/plain
    accept-encoding: gzip
    Content-Length: 972
[2024-05-08T14:02:33Z DEBUG ureq::response] Body entirely buffered (length: 728)
[2024-05-08T14:02:33Z DEBUG ureq::pool] adding stream to pool: http|localhost|8888 -> Stream(TcpStream { addr: [::1]:60872, peer: [::1]:8888, fd: 12 })
[2024-05-08T14:02:33Z DEBUG ureq::unit] response 200 to POST http://localhost:8888/?v=1&additionalfeeoutputindex=1&maxadditionalfeecontribution=68&minfeerate=1
Sent fallback transaction txid: 1c8ecc31b4a4b810170aa8fc832e8d11fe82b9625a2e5ec3dd8eb5f536e4d055
Sent fallback transaction hex: 02000000000102f737e844fc219ac5a7a9852d40b152d633b5b84e377bfdde2484d577db1899970100000000fdffffff325127ba0a20c42faff927028d661c5bf0e44750fe9af57c8d7fde4578f7abbf0100000000fdffffff021027000000000000160014f02df32ec108dc3986b63b2826a925c60181f7bd7025000000000000160014fcee2a3b4a79a735897cd8da8b0ae089125e7aec02473044022006b5fb1c85a37118a5c9d7e401569cebf14b9398c8b235fdb7a7fada71d8d122022019ff1a9a38e7b4a4d97eeba1244e7e07fad6239627f09de7de5844f4ab44c6bb0121037ce0dcfcbb10ba6115fac9e066bc4d8627189c36b27c5081d784c9161f79eda102473044022010d9118997f74948e035b3f665481611e0b3cdc458536b78e8ba125749f8e73702206a92e2480dcb313ab24c257e589b70ee08ba66c85d2e2df0b0fb3b368bcbb616012103cee664ab34e780b7b7b1ca0f2a31e40d785d0f9ed5db55bfecab9405d1ce301400000000
[2024-05-08T14:02:33Z DEBUG payjoin::send] psbt: "cHNidP8BAMMCAAAAA/c36ET8IZrFp6mFLUCxUtYztbhON3v93iSE1XfbGJmXAQAAAAD9////MlEnugogxC+v+ScCjWYcW/DkR1D+mvV8jX/eRXj3q78BAAAAAP3///+VJv46X9f8bAcgoCsn6iZrmvFHXS2tDzLwmmwHclguJgAAAAAA/f///wL17QwAAAAAABYAFPAt8y7BCNw5hrY7KCapJcYBgfe9LCUAAAAAAAAWABT87io7SnmnNYl82NqLCuCJEl567AAAAAAAAQCaAgAAAAJVOXyCUb0/VFeWJDlHpCQMD3DdOSukM7PPG6flIrx8tQEAAAAA/f////yYud4fRLplvAqPdVoPjj4/QrZtJEBU72NfZPA4dtQ9AAAAAAD9////AhAnAAAAAAAAFgAUBlTIYWgMPk7BaOJBD2cRwp1oCwVAJgAAAAAAABYAFOGLhlQOf0ld/YhofaZba0PLzmOUAAAAAAEBH0AmAAAAAAAAFgAU4YuGVA5/SV39iGh9pltrQ8vOY5QAAAEBH+XGDAAAAAAAFgAUrnw7xvJ/4mPOOAxCj+8c66Kl/TsBCGsCRzBEAiAaKFI6ZgQjlDaOU1i8nPgOzcoOWVpo10nxEYaBdd1eDwIgGduddqJdvMvJecfMAt44wH/hrUJEVBi6Z3YB6nMMCXMBIQMIbj5gipd6d+kMAWru3b0aSQOVQiGuOqcud9N/mkNOpAAAAA=="
[2024-05-08T14:02:33Z DEBUG payjoin_cli::app::v1] Error processing response: Validation(ValidationError { internal: SenderTxinContainsNonWitnessUtxo })
[2024-05-08T14:02:33Z DEBUG ureq::stream] dropping stream: Stream(TcpStream { addr: [::1]:60872, peer: [::1]:8888, fd: 12 })
Error: Failed to process response The receiver sent an invalid response: an input in proposed transaction belonging to the sender contains non-witness UTXO information

My payjoin-cli sender is selecting multiple inputs, which appears to be unsupported by JoinMarket . JM is only clearing the utxo fields for one sender input.

spacebear21 avatar May 08 '24 14:05 spacebear21

I made a successful payjoin tx with the above patch in JoinMarket. Signet tx here (notice the 3 inputs)

spacebear21 avatar May 08 '24 15:05 spacebear21

Nice catch and patch on the JoinMarket side. Slick proxying to get it working for this test.

I wonder if JM code handles fee calculation for which party owes what when multiple inputs are assigned, or if the assumption that only 1 sender input is used affects any other part of the JM code than what you've changed.

DanGould avatar May 08 '24 17:05 DanGould

This is closed by @grizznaut from fixes in #241 and https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/1700

Big thank you for championing this issue

DanGould avatar May 30 '24 17:05 DanGould