Transaction rejection (consensus + relay rules)

[sr-gi]

After getting a serialised transaction from the Watcher (and therefore decrypting the encrypted_blob) a responder can face multiple rejections from bitcoind when trying to broadcast the justice transaction:

RPC_VERIFY_REJECTED (-26)

This include most of, both, consensus and relay rules. Invalid transactions (properly formatted but invalid, like unsigned) fit here too. This is an exhaustive¹ list errors.

Codes REJECT_MALFORMED 0x01 (1) REJECT_INVALID 0x10 (16) REJECT_OBSOLETE 0x11 (17) REJECT_DUPLICATE 0x12 (18) REJECT_NONSTANDARD 0x40 (64) REJECT_INSUFFICIENTFEE 0x42 (66) REJECT_CHECKPOINT 0x43 (67) REJECT_HIGHFEE 0x100 (256)

Relay rules

TX_MEMPOOL_POLICY

• mempool min fee not met, REJECT_INSUFFICIENTFEE
• min relay fee not met, REJECT_INSUFFICIENTFEE
• txn-mempool-conflict, REJECT_DUPLICATE
• too-long-mempool-chain, REJECT_NONSTANDARD ²
• insufficient fee (rejecting replacement), REJECT_INSUFFICIENTFEE (3 instances of this in the code)
• too many potential replacements, REJECT_NONSTANDARD ²
• replacement-adds-unconfirme, REJECT_NONSTANDARD ²
• mempool full, REJECT_INSUFFICIENTFEE ³

TX_NOT_STANDARD

• reason, REJECT_NONSTANDARD ⁴
• tx-size-small, REJECT_NONSTANDARD
• txn-already-known, REJECT_DUPLICATE ²
• bad-txns-nonstandard-inputs, REJECT_NONSTANDARD
• bad-txns-too-many-sigops, REJECT_NONSTANDARD
• absurdly-high-fee, REJECT_HIGHFEE ³
• non-mandatory-script-verify-flag, REJECT_NONSTANDARD

TX_PREMATURE_SPEND

• non-final, REJECT_NONSTANDARD
• non-BIP68-final, REJECT_NONSTANDARD ²

TX_CONFLICT

• txn-already-in-mempool, REJECT_DUPLICATE ²
• txn-already-know, REJECT_DUPLICATE ²

TX_WITNESS_MUTATED

• bad-witness-nonstandard, REJECT_NONSTANDARD
• reason, reject_reason (everything is parametrised in this one, line 932) ²

Consensus rules

• coinbase, REJECT_INVALID
• bad-txns-spends-conflicting-tx, REJECT_INVALID
• mandatory-script-verify-flag-failed, REJECT_INVALID

Some of the aforementioned relay rules can be attack surfaces. ~~For instance a transaction paying too much fees is a valid transaction, but bitcoind would reject. Furthermore, if we modify our node and send it to the network, Bitcoin Core nodes would drop (and not propagate) it. Check this for the hard imposed Bitcoin Core max fee.~~ This is actually not a relay policy but a wallet one, meaning that this transactions will be relayed if they are accepted by our local node (by changing -maxfee for instance).

A similar thing may happen to transactions including dust outputs, or anything in ⁴.

RPC_VERIFY_ERROR (-25)

This seems to only cover a transaction trying to spend from non-existing / already spent outputs:

• Missing inputs.

However, RPC_TRANSACTION_ERROR aliases RPC_VERIFY_ERROR and it's the default return for RPCErrorFromTransactionError [ref]

RPC_VERIFY_ALREADY_IN_CHAIN (-27)

The transaction is already in the blockchain. Either another tower, the customer or who-knowns has already broadcast the transaction.

Nothing to worry about here, we can get the confirmation count and monitor the transaction until the end of the appointment.

RPC_DESERIALIZATION_ERROR (-22)

The transaction cannot be deserialised. In other words, the transaction is malformed. Probably random data sent as transaction data. The Responder SHOULD NEVER face this issue, since the Watcher checks that the transaction properly deserialises before handing the job.

Finally, there is an additional rejection problem I haven't been able to test myself yet:

RPC_CLIENT_NOT_CONNECTED (-9)

This error relates to the node not being connected to any other node.

Some of this errors (like the later or the ones in ⁴) may not be raised in regtest or testnet.

Leaving this here both to discuss what to do with relay rules and as notes of all the possible errors.

¹ The list has been manually parsed from bitcoin/src/validation.cpp up to the first rejection referring to blocks. ² Have to check this ones properly. Not sure about when they're triggered. ³ Attack surface. ⁴ Any of the reasons that make a transaction non-standard.

[stonecoldpat]

The main problem would be errors that prevent the real transaction getting delivered. i.e. if the attacker can give our watchers a transaction that double-spends it, but that double-spend never gets in and we never relay the real transaction.

We should really be trying to connect to 1000+ nodes on the network and bypassing our own memory pool to relay transactions.

What is also important to consider are the "ban peer" rules which I don't think is referenced here. Is there any special tricks someone could pull off that would make us broadcast a transaction and get us banned.

[sr-gi]

The main problem would be errors that prevent the real transaction getting delivered. i.e. if the attacker can give our watchers a transaction that double-spends it, but that double-spend never gets in and we never relay the real transaction.

I'm guessing that i.e should be e.g and you're given an example instead of referring to the only reason why that can happen. Anyway, I'm no getting your example. If the Watcher receives a double-spend in the appointment it means that the conflicting transaction is already in. Otherwise it is a mempool conflict and that can be solved in a different way.

Moreover, literally every error in RPC_VERIFY_REJECTED falls into the category you're mentioning (errors that prevent the real transaction getting delivered)

We should really be trying to connect to 1000+ nodes on the network and bypassing our own memory pool to relay transactions.

Connecting to more nodes may not solve the problem. If a transaction is non-standard, and assuming a bast majority of nodes running Bitcoin Core software, the transaction will be dropped before reaching a miner.

What is also important to consider are the "ban peer" rules which I don't think is referenced here. Is there any special tricks someone could pull off that would make us broadcast a transaction and get us banned.

I have to dig further into this, but I don't think this case is that serious. Transactions that may make our peers increase our banscore are the ones that fall into breaking the consensus rules, but our node will reject those and not propagate them, so if sending transaction through bitcoind this shouldn't be a problem (still to verify to what extend this claim is true).

On the other hand, and as I was mentioning in the original text, non-standard transaction are. For instance:

If TEOS accepts an appointment that does not break any consensus rule, but it does break a relay rule (let's say the fee too high one), the likelihood of such a transaction making it into the blockchain is low, but the transaction itself is valid. It's not the customers fault that we have (and most of the network has) a different set of more restrictive acceptance rules.

I feel that, in order to avoid this kind of problems, the appointment should acknowledge wether the transaction is standard or not, in the same way it should acknowledge the fees and so on. That way we would have evidence that this is not our fault is this ends up happening.

[bigspider]

I feel that, in order to avoid this kind of problems, the appointment should acknowledge wether the transaction is standard or not, in the same way it should acknowledge the fees and so on. That way we would have evidence that this is not our fault is this ends up happening.

Not sure I follow. In bitcoin, our signature is nothing more than a signed receipt. Writing some kind of acknowledgment in the appointment or in the ToS of our service doesn't seem to make any practical difference? We can always say "hey, look, our ToS say that the transaction has to be standard and your transaction wasn't".

More generally, this could apply to other failures as well. We try to relay the transaction, and we log any error that we get. While we clearly need to distinguish what errors are situations out of our control and what are not, there is no smart contract that they can use to slash us, so solving it "the old way" should be good enough?

[bigspider]

(of course this does not apply to double-spend attacks mentioned above, we should make sure to consider those)

[sr-gi]

I feel that, in order to avoid this kind of problems, the appointment should acknowledge wether the transaction is standard or not, in the same way it should acknowledge the fees and so on. That way we would have evidence that this is not our fault is this ends up happening.

Not sure I follow. In bitcoin, our signature is nothing more than a signed receipt. Writing some kind of acknowledgment in the appointment or in the ToS of our service doesn't seem to make any practical difference? We can always say "hey, look, our ToS say that the transaction has to be standard and your transaction wasn't".

Check https://github.com/PISAresearch/pisa/blob/master/12-watchtower-API.md

It is true that we cannot enforce it via a smart contract, but the approach is having evidence of agreement.

More generally, this could apply to other failures as well. We try to relay the transaction, and we log any error that we get. While we clearly need to distinguish what errors are situations out of our control and what are not, there is no smart contract that they can use to slash us, so solving it "the old way" should be good enough?

It would be different to have errors that depends on us (e.g. we missed to respond, were not properly connected, missed a reorg, ...), that other that directly depend on the received data.

[sr-gi]

Continue digging into this, we can rule out all the non-transitory errors before reaching the Responder by using testmempoolaccept. That means that all errors regarding the transaction breaking a consensus / standardness rule that cannot be fixed without recreating the transaction will be out of the picture.

This leaves us with:

Relevant

• mempool min fee not met

The minimum fee for the mempool is not met, this can change over time, so waiting may fix it.

• txn-already-in-mempool

The transaction is already in or mempool, we need to keep watching until it gets IRREVOCABLY_RESOLVED.

• txn-mempool-conflict

There's a conflict with a transaction in the mempool. There no much we can do here but retry in case the transaction in the mempool gets evicted.

• mempool full

The mempool is full. Waiting may solve this.

• non-final

The transaction we're trying to send has a nLockTime that has not been reached yet. Retrying later on may fix this.

• non-BIP68-final

The transaction we're trying to send has a relative time-lock that has not been met (CSV). Same as the previous one.

• txn-already-known

The transaction is already in the blockchain. Same applies as for transaction being in the mempool, we should monitor them until IRREVOCABLY_RESOLVED

Relevant (once we can fee bump)

• min relay fee not met

Fee too low. Bump.

• absurdly-high-fee

We should not hit this one provided our node is set to allow high fees. It's also worth considering that the fee margin the tower may be rather small, otherwise it would be uneconomical to bump.

• insufficient fee

Fee too low. Bump.

Not relevant (non-transitory errors, cannot be fixed without resigning)

• tx-size-small

The transaction is too small.

• bad-txns-nonstandard-inputs

The transaction has non-standard ScriptSigs

• bad-txns-too-many-sigops

The tx exceed the SigOps limit.

• non-mandatory-script-verify-flag

At least one of the tx scripts break some of the standardness rules.

• bad-witness-nonstandard

At least one of the provided witnesses is not standard.

• coinbase

The transaction is a coinbase transaction (outside a block generation)

• mandatory-script-verify-flag-failed

At least one of the scripts break the consensus rules.

• bad-txns-spends-conflicting-tx

Trying to spend from an output that would not exists if this tx is created. e.g: RBF txA for txB which spends outputs from txA.

To check (includes RBFs)

• too-long-mempool-chain

Too many unconfirmed parents, tx spends from big chain of unconfirmed txs (defaults to 25).

This is transitory, since confirming some of the parents may fix it. It should not apply to LN but may to other protocols.

May be relevant.

• too many potential replacements

Transaction tries to replace too many transactions in the mempool.

This would be transitory as long as the other transaction on the mempool are evicted for whatever reason, even though it seems quite unlikely.

May be relevant.

• replacement-adds-unconfirmed

A replacement has a new input pointing to a tx in the mempool.

This is transitory, the unconfirmed parent can be received after the transaction is being rejected.

Relevant.

• bad-txns-inputs-missingorspent

Missing inputs. This actually returns missing-inputs as reject-reason.

This may be relevant since the transaction to be relayed may be currently missing an input that may be received in the future.

May be relevant.

[darosior]

Relevant (once we can fee bump)

Doesn't mempool min fee not met and mempool full belong here too ?

[sr-gi]

Relevant (once we can fee bump)

Doesn't mempool min fee not met and mempool full belong here too ?

They do, but they can also be solved without anchor outputs atm by just waiting, as opposed to min relay fee not met since that one is not dynamically increased AFAIK, is it?