extension-bolt: taproot gossip (features 32/33)

[ellemouton]

Overview

The initial version of the Lightning Network gossip protocol as defined in BOLT 7 was designed around P2WSH funding transactions. For these channels, the channel_announcement message is used to advertise the channel to the rest of the network. Nodes in the network use the content of this message to prove that the channel is sufficiently bound to the Lightning Network context and that it is owned by the nodes advertising the channel. This proof and verification protocol is, however, not compatible with SegWit V1 (P2TR) outputs and so cannot be used to advertise the channels defined in the Simple Taproot Channel proposal. This document thus aims to define an updated gossip protocol that will allow nodes to both advertise and verify taproot channels. This part of the update affects the announcement_signatures and channel_announcement messages.

The opportunity is also taken to rework the node_announcement and channel_update messages to take advantage of BIP-340 signatures and TLV fields. Timestamp fields are also updated to be block heights instead of Unix timestamps.

[ellemouton]

Thanks to @rustyrussell , @joostjager & @GeorgeTsagk for the initial set of comments on the pre-draft PR! I will carry over some of the comments over to this PR so that a discussion can take place :)

[ellemouton]

note from @rustyrussell - would be good to support legacy updates in the new messages in order to take advantage of the new blockheight fields. This gives us better Spam prevention & plays better with minisketch set reconciliation

[ellemouton]

Hey everyone! Was planning on having this doc updated by today but got side tracked a bit. Will aim to have the updated changes pushed by the time of the next bi-monthly meeting.

In the meantime, here is a summary of what I plan to update as a result of the discussions from the NYC meeting. Let me know if something looks off.

1. Plan for a future where the script can be anything

Instead of requiring the channel peers to specify the two bitcoin keys in the channel_announcement_, we will instead only require them to provide a signature for the output key that is found on-chain. This means that once nested MuSig2 has a security proof and has been spec'd out, then the signature in the channel_announcement_2 can be for the key:

P_agg = MuSig2.KeyAgg(MuSig2.KeySort(`node_id_1`, `node_id_2`, `taproot_output_key`))

So to cater for this future, we can say that channel_annoucement_2 verifiers should do the following:

• if there are no bitcoin_key_* fields in the announcement:

  • they can calculate the 3-of-3 P_agg as specified above and validate the sig accordingly.

• else, bitcoin_key_* fields are in the announcements (tlv) along with an optional merkle_root in which case the P_agg is:

  _agg = MuSig2.KeyAgg(MuSig2.KeySort(`node_id_1`, `node_id_2`, `bitcoin_key_1`, `bitcoin_key_2`))
  

  and the verifier also just checks that they can reconstruct what is found on-chain at the given SCID:

  _btc_agg = MuSig2.KeyAgg(MuSig2.KeySort(`bitcoin_key_1`, `bitcoin_key_2`))
  
  _output = P_btc_agg + tagged_hash("TapTweak", P_btc_agg || merkle_root_hash)
  
  _output =? taproot_output
  

For the signers, this would mean the following:

1. they can switch to using nested MuSig2 (or any other construction for that matter) at anytime as long as they can produce a valid partial sig for it.
2. The default option for now would be to follow what is spec'ed in this doc which for now will be a flat 4-of-4 MuSig2 using node_id_1, node_id_2, bitcoin_key_1 and bitcoin_key_2.

The verifier will be able to verify both types from the get go.

2. Plan for a future where not every channel announcement contains UTXO proof

We want to allow the flexibility so that in future we dont need to tie every channel to a UTXO but instead use the rule of "here is a channel without a proof, please look at all the other open channels I have announced with a proof and use that to determine if I do infact have the allowance to open one without"

What I took away from the meeting is that to create a sort of carve-out for that future, what we will do today is to start putting the channel capacity in the channel_announcement and the verifier for now just checks that this amount is less than or equal to the output on-chain.

Let me know if I missed something here.

3. Message structures

New type definitions:

• bip340_sig: a 64-byte bitcoin Elliptic Curve Schnorr signature as per [BIP-340][bip-340].
• boolean_tlv: a zero length TLV record. If the TLV is present then true is implied, otherwise false is implied.
• partial_signature: a 32-byte partial MuSig2 signature as defined in [BIP 327][bip-327].
• public_nonce: a 66-byte public nonce as defined in [BIP 327][bip-327].
• utf8: a byte as part of a UTF-8 string. A writer MUST ensure an array of these is a valid UTF-8 string, a reader MAY reject any messages containing an array of these which is not a valid UTF-8 string.

announcement_signatures_2 (no tlv here)

1. type 260
2. data:
   • [channel_id:channel_id]
   • [short_channel_id:short_channel_id]
   • [partial_signature:partial_signature]

channel_announcement_2

1. type 267

2. data:

   • [bip340_sig:bip340_sig]
   • [channel_announcement_2_tlvs:tlvs]

3. type: 0 (chain_hash)

4. data:

   • [chain_hash:chain_hash]

5. type: 2 (features)

6. data:

   • [...*byte: features]

7. type: 4 (short_channel_id)

8. data:

   • [short_channel_id:short_channel_id]

9. type: 6 (node_id_1)

10. data:

    • [point:point]

11. type: 8 (node_id_2)

12. data:

• [point:point]

1. type: 10 (bitcoin_key_1)
2. data:

• [point:point]

1. type: 12 (bitcoin_key_2)
2. data:

• [point:point]

1. type: 14 (merkle_root_hash)
2. data:
   • [32*byte:hash]

node_announcement_2

1. type 269

2. data:

   • [bip340_sig:bip340_sig]
   • [node_announcement_2_tlvs:tlvs]

3. tlv_stream: node_announcement_2_tlvs

4. types:

   1. type: 0 (features)
   2. data:
      • [...*byte: features]
   3. type: 2 (block_height)
   4. data:
      • [u32: block_height]
   5. type: 4 (node_id)
   6. data:
      • [point:node_id]
   7. type: 1 (color)
   8. data:
      • [rgb_color:rgb_color]
   9. type: 3 (alias)
   10. data:
       • [...*utf8:alias]
   11. type: 5 (ipv4_addrs)
   12. data:
       • [...*ipv4_addr: ipv4_addresses]
   13. type: 7 (ipv6_addrs)
   14. data:
       • [...*ipv6_addr: ipv6_addresses]
   15. type: 9 (tor_v3_addrs)
   16. data:
       • [...*tor_v3_addr: tor_v3_addresses]
   17. type: 11 (dns_hostnames)
   18. data:
       • [...*dns_hostname: dns_hostnames]

The following subtypes are defined:

1. subtype: rgb_color

2. data:

   • [byte:red]
   • [byte:green]
   • [byte:blue]

3. subtype: ipv4_addr

4. data:

   • [u32:addr]
   • [u16:port]

5. subtype: ipv6_addr

6. data:

   • [16*byte:addr]
   • [u16:port]

7. subtype: tor_v3_addr

8. data:

   • [35*utf8:onion_addr]
   • [u16:port]

9. subtype: dns_hostname

10. data:

    • [u16:len]
    • [len*utf8:hostname]
    • [u16:port]

channel_update_2

1. type 271

2. data:

   • [bip340_sig:bip340_sig]
   • [channel_update_2_tlvs:tlvs]

3. tlv_stream: channel_update_2_tlvs

4. types:

   1. type: 0 (chain_hash)
   2. data:
      • [chain_hash:chain_hash]
   3. type: 2 (short_channel_id)
   4. data:
      • [short_channel_id:short_channel_id]
   5. type: 4 (block_height)
   6. data:
      • [u32:block_height]
   7. type: 6 (disable)
   8. data:
      • [boolean_tlv:disable]
   9. type: 8 (direction)
   10. data:
       • [boolean_tlv:direction]
   11. type: 10 (cltv_expiry_delta)
   12. data:
       • [u16:cltv_expiry_delta]
   13. type: 12 (htlc_minimum_msat)
   14. data:
       • [u64:htlc_minimum_msat]
   15. type: 14 (htlc_maximum_msat)
   16. data:
       • [u64:htlc_maximum_msat]
   17. type: 16 (fee_base_msat)
   18. data:
       • [u32:fee_base_msat]
   19. type: 18 (fee_proportional_millionths)
   20. data:
       • [u32:fee_proportional_millionths]

4. Legacy channel info in the new messages?

1. It makese sense to adertise both node_announcement_2 and node_annoucement for a while
2. Doesnt really make sense to convert old chan announcements to the new message (would also be difficult to do since would need to resign)
3. Makes sense to use new channel_update_2 for legacy channels so we can get the easier set reconcilliation benifits due to using block height instead of timestamp.

[TheBlueMatt]

type: 6 (disable)

Can we define a separate "my peer is offline/I can't forward or receive anything" bit from "I wont forward anything out on this channel/I don't have enough capacity to send, but I can receive". Could be a separate TLV or just make it one disable_flags TLV.

[ellemouton]

am finally fully focused on this so will push updates here sometime next week 👍

[ellemouton]

Alright - pushed some changes incorporating the changes mentioned here along with @TheBlueMatt s disable flags suggestion

Still need to do a loooot of fleshing out (see all comments/questions marked TODO) but this is a good time to start getting more feedback I think

[rustyrussell]

First pass just from a gossip POV:

1. In bolt12 we used bip340sig instead of bip340_sig, like other fundamental types. Bikeshed time?
2. Boolean tlv is awkward, can we just name the field second_peer?
3. MsgHash should use fieldname, not type name. And I suggest renaming to signature everywhere anyway, since context is always present.
4. type headers fixed so they are parsed correctly by tools/extract-formats.py

No doubt more feedback as we actually implement, this was just from trying to import spec into our code!

diff --git a/bolt-taproot-gossip.md b/bolt-taproot-gossip.md
index b4fbcd5..a9bf523 100644
--- a/bolt-taproot-gossip.md
+++ b/bolt-taproot-gossip.md
@@ -254,7 +254,7 @@ While the network is in the upgrade phase, the following suggestions apply:
 
 The following convenient types are defined:
 
-* `bip340_sig`: a 64-byte bitcoin Elliptic Curve Schnorr signature as
+* `bip340sig`: a 64-byte bitcoin Elliptic Curve Schnorr signature as
   per [BIP-340][bip-340].
 * `partial_signature`: a 32-byte partial MuSig2 signature as defined
   in [BIP-MuSig2][bip-musig2].
@@ -262,8 +262,6 @@ The following convenient types are defined:
 * `utf8`: a byte as part of a UTF-8 string. A writer MUST ensure an array of
   these is a valid UTF-8 string, a reader MAY reject any messages containing an
   array of these which is not a valid UTF-8 string.
-* `boolean_tlv`: a zero length TLV record. If the TLV is present then true is
-  implied, otherwise false is implied.
 
 ### Feature Bits
 
@@ -350,7 +348,7 @@ The recipient:
         - SHOULD ignore the message.
     - Otherwise:
         - MUST store the nonces so that they can be used for the partial signature
-          construction required for constructing the `announcement_signatures`
+          construction required for constructing the `announcement_signatures_2`
           message.
         - If it has not yet done so, SHOULD respond with its own `channel_ready`
           message with the nonce fields set.
@@ -390,7 +388,7 @@ A node:
     - MUST set the `partial_signature` field to the 32-byte `partial_sig` value
       of the partial signature calculated as described in [Partial Signature
       Calculation](#partial-signature-calculation). The message to be signed is
-      `MsgHash("channel_announcement", "announcement_sig", m)` where `m` is the
+      `MsgHash("channel_announcement", "signature", m)` where `m` is the
       serialisation of the `channel_announcement_2` message tlv stream (see the
       [`MsgHash`](#signature-message-construction) definition).
 - otherwise:
@@ -442,7 +440,7 @@ channel.
 
 1. type: 267 (`channel_announcement_2`)
 6. data:
-    * [`bip340_sig`:`announcement_signature`]
+    * [`bip340sig`:`signature`]
     * [`channel_announcement_2_tlvs`:`tlvs`]
 
 1. `tlv_stream`: `channel_announcement_2_tlvs`
@@ -494,9 +492,9 @@ Unlike the legacy `node_announcement` message, this message makes use of a
 BIP340 signature instead of an ECDSA one. This will allow nodes to be backed
 by multiple keys since MuSig2 can be used to construct the single signature.
 
-1. type 269
+1. type: 269 (`node_announcement_2`)
 2. data:
-    * [`bip340_sig`:`announcement_sig`]
+    * [`bip340sig`:`signature`]
     * [`node_announcement_2_tlvs`:`tlvs`]
 
 1. `tlv_stream`: `node_announcement_2_tlvs`
@@ -560,8 +558,8 @@ The following subtypes are defined:
 
 #### Message Field Descriptions
 
-- `bip340_sig` is the [BIP340][bip-340] signature for the `node_id` key. The
-  message to be signed is `MsgHash("node_announcement_2", "bip340_sig", m)`
+- `bip340sig` is the [BIP340][bip-340] signature for the `node_id` key. The
+  message to be signed is `MsgHash("node_announcement_2", "signature", m)`
   where `m` is the serialised TLV stream (see the
   [`MsgHash`](#signature-message-construction) definition).
 - `features` is a bit vector with bits set according to [BOLT #9](09-features.md#assigned-features-flags)
@@ -594,11 +592,11 @@ ok to send both? if so - what if the info inside them differs?
 The sender:
 
 - MUST set TLV fields 0, 2 and 4.
-- MUST set `announcement_sig` to a valid [BIP340][bip-340] signature for the
+- MUST set `signature` to a valid [BIP340][bip-340] signature for the
   `node_id` key. The message to be signed is
-  `MsgHash("node_announcement_2", "announcement_sig", m)` where `m` is the
+  `MsgHash("node_announcement_2", "signature", m)` where `m` is the
   serialisation of the `node_announcement_2` message excluding the
-  `announcement_sig` field (see the
+  `signature` field (see the
   [`MsgHash`](#signature-message-construction) definition).
 - MAY set `color` and `alias` to customise appearance in maps and graphs.
 - If the node sets the `alias`:
@@ -635,7 +633,7 @@ The receiver:
     - SHOULD send a `warning`.
     - MAY close the connection.
     - MUST NOT process the message further.
-- if `announcement_sig` is NOT a valid [BIP340][bip-340] signature (using
+- if `signature` is NOT a valid [BIP340][bip-340] signature (using
   `node_id` over the message):
     - SHOULD send a `warning`.
     - MAY close the connection.
@@ -692,9 +690,9 @@ value for `amount_msat` and the minimal value for `cltv_expiry`, to be used for
 the last HTLC in the route, are provided in the payment request
 
 (see [BOLT #11][[bolt-11-tagged-fields]]).
-1. type 271
+1. type: 271 (`channel_update_2`)
 2. data:
-    * [`bip340_sig`:`bip340_sig`]
+    * [`bip340sig`:`signature`]
     * [`channel_update_2_tlvs`:`tlvs`]
 
 1. `tlv_stream`: `channel_update_2_tlvs`
@@ -711,9 +709,7 @@ the last HTLC in the route, are provided in the payment request
     1. type: 6 (`disable_flags`)
     2. data:
         * [`byte`:`disable_flags`]
-    1. type: 8 (`direction`)
-    2. data:
-        * [`boolean_tlv`:`direction`]
+    1. type: 8 (`second_peer`)
     1. type: 10 (`cltv_expiry_delta`)
     2. data:
         * [`u16`:`cltv_expiry_delta`]
@@ -757,10 +753,9 @@ peer is offline.
 - The `disable` bit field can be used to advertise to the network that a channel
   is disabled and that it should not be used for routing. The individual
   `disable_flags` bits can be used to communicate more fine-grained information.
-- The `direction` is used to indicate which node in the channel node pair
-  has created and signed this message. If the node was `node_id_1` in the
-  `channel_announcment` message then the `direction` is 0 (false) otherwise
-  it is 1 (true) and the node is `node_id_2` in the `channel_announcement`
+- The `second_peer` is used to indicate which node in the channel node pair
+  has created and signed this message. If present, the node was `node_id_2` in the
+  `channel_announcment`, otherwise the node is `node_id_1` in the `channel_announcement`
   message.
 - `cltv_expiry_delta` is the number of blocks that the node will subtract from
   an incoming HTLC's `cltv_expiry`.
@@ -778,14 +773,13 @@ peer is offline.
 
 #### TLV Defaults
 
-For types 0, 6, 8, 10, 12, 14, 16 and 18, the following defaults apply if the
+For types 0, 6, 10, 12, 14, 16 and 18, the following defaults apply if the
 TLV is not present in the message:
 
 | `channel_update_2` TLV Type        | Default Value                                                      | Comment                                                                                     |
 |------------------------------------|--------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
 | 0  (`chain_hash`)                  | `6fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d6190000000000` | The hash of the genesis block of the mainnet Bitcoin blockchain.                            | 
 | 6  (`disable`)                     | empty                                                              |                                                                                             | 
-| 8  (`direction`)                   | false                                                              |                                                                                             | 
 | 10 (`cltv_expiry_delta`)           | 80                                                                 |                                                                                             | 
 | 12 (`htlc_minimum_msat`)           | 1                                                                  |                                                                                             | 
 | 14 (`htlc_maximum_msat`)           | floor(channel capacity / 2)                                        | // TODO: remove this since makes encoding/decoding dependent on things outside the message? | 
@@ -817,9 +811,9 @@ The origin node:
     - MUST set `chain_hash` AND `short_channel_id` to match the 32-byte hash AND
       8-byte channel ID that uniquely identifies the channel specified in the
       `channel_announcement_2` message.
-- MUST set `bip340_sig` to a valid [BIP340][bip-340] signature for its own
+- MUST set `signature` to a valid [BIP340][bip-340] signature for its own
   `node_id` key. The message to be signed is
-  `MsgHash("channel_update_2", "bip340_sig", m)` where `m` is the serialised
+  `MsgHash("channel_update_2", "signature", m)` where `m` is the serialised
   TLV stream of the `channel_update` (see the
   [`MsgHash`](#signature-message-construction) definition).
 - SHOULD NOT create redundant `channel_update_2`s.
@@ -834,7 +828,7 @@ The receiving node:
       channels.
 - SHOULD accept `channel_update_2`s for its own channels (even if non-public),
   in order to learn the associated origin nodes' forwarding parameters.
-- if `bip340_sig` is NOT a valid [BIP340][bip-340] signature (using `node_id`
+- if `signature` is NOT a valid [BIP340][bip-340] signature (using `node_id`
   over the message):
     - SHOULD send a `warning` and close the connection.
     - MUST NOT process the message further.
@@ -890,8 +884,7 @@ aggnonce = Musig2.NonceAgg(announcement_node_secnonce_1, announcement_bitcoin_pu

The message, msg that the peers will sign is the serialisation of -channel_announcement_2 without the announcement_sig field (i.e. without -type 0) +channel_announcement_2 without the signature field (i.e. only the TLV)

With all the information mentioned, both peers can now construct the [Session Context][musig-session-ctx] defined by the MuSig2 protocol which is @@ -1004,7 +997,7 @@ The full list of inputs required:

• tr_output_key
• msg: the serialised channel_announcement_2 tlv stream.
• sig: the 64-byte BIP340 signature found in the
• channel_announcement_signature field of the channel_announcement_2

• signature field of the channel_announcement_2 message. This signature must be parsed into R and s values as defined in BIP327.

@@ -1016,7 +1009,7 @@ P_agg = MuSig2.KeyAgg(MuSig2.KeySort(node_id_1, node_id_2, tr_output_key))

The signature can then be verified as follows:

• Let pk = bytes(P_agg) where the bytes function is defined in BIP340. -- Let m = MsgHash("channel_announcement_2", "announcement_signature", msg) +- Let m = MsgHash("channel_announcement_2", "signature", msg)
• Use the BIP340 Verify function to determine if the signature is valid by passing in pk, m and sig.

@@ -1060,7 +1053,7 @@ The full list of inputs required:

• bitcoin_key_2
• msg: the serialised channel_announcement_2 tlv stream.
• sig: the 64-byte BIP340 signature found in the
• channel_announcement_signature field of the channel_announcement_2

• signature field of the channel_announcement_2 message. This signature must be parsed into R and s values as defined in BIP327.

@@ -1072,7 +1065,7 @@ P_agg = MuSig2.KeyAgg(MuSig2.KeySort(node_id_1, node_id_2, bitcoin_key_1,

The signature can then be verified as follows:

• Let pk = bytes(P_agg) where the bytes function is defined in BIP340. -- Let m = MsgHash("channel_announcement_2", "announcement_signature", msg) +- Let m = MsgHash("channel_announcement_2", "signature", msg)
• Use the BIP340 Verify function to determine if the signature is valid by passing in pk, m and sig.

[rustyrussell]

Note: feature bit assignment clash with #1036 ! Someone needs to move?

(This is why we put feature bits in the title, for easy disambiguation).

[t-bast]

I closed #1036 for now to free up that feature bit, taproot gossip is much more important to get as soon as we can ;)

[ellemouton]

Thanks for the feedback (added the suggested diff - thanks @rustyrussell!) ! A few notes/questions for discussion in the upcoming meeting:

1. Re adding a permanent bit to the disabled flags: Today, we see a channel closure on-chain and can safely delete that channel from our graph. If we have this permanent bit - what do we do in the case of the channel not actually being closed on-chain? We cannot delete the channel data since the channel nodes could send a new channel update with permanent not set. So we'd need to persist a log of: "channel ID -> closed" so that we throw away any updates after receiving one that indicates permanent failure. I guess this is ok as it is a nice set up for Gossip 2.0 where we would need to do that anyway.

2. The other thing I think still seems a big fuzzy to me is exactly how we want to handle both existing legacy channels along with new legacy channels (ie, new legacy channels between 2 nodes that support taproot gossip):

   • For existing, legacy channels: - If both peers update to support option_taproot_gossip, do we want to have a protocol for re-negotiation & re-signing a channel_announcement_2 message? I dont think I see a benefit of doing this. - If the individual peers update to option_taproot_gossip, should they start sending out both old and new channel_update? I assume yes. The tricky part here is 1) When can they stop announcing the old one? 2) for nodes that understand both, how do they know which one is newest (they may have conflicting policy values & one uses timestamp while the other uses blockheight) ? should they still re-propagate both?
   • For new, legacy channels where both peers support option_taproot_gossip: - should they negotiate both an old and new channel_announceent? again I assume we can just stick to the old one - same question re the channel_update as stated above in the existing-legacy case.

[TheBlueMatt]

Today, we see a channel closure on-chain and can safely delete that channel from our graph. If we have this permanent bit - what do we do in the case of the channel not actually being closed on-chain? We cannot delete the channel data since the channel nodes could send a new channel update with permanent not set. So we'd need to persist a log of: "channel ID -> closed" so that we throw away any updates after receiving one that indicates permanent failure. I guess this is ok as it is a nice set up for Gossip 2.0 where we would need to do that anyway.

This didn't manage to come up at the meeting, but I'm not sure its worth doing anything here. Yes, someone could send a closed update, cause you to remove it from your gossip store, then go get you to re-add it later, but...so what? They're misbehaving and gonna have people not route through their channel, that's their problem. Sure, you don't want gossip to flap causing lots of traffic on the network, but that can be addressed by just keeping a short-term cache (as long as your usual gossip rate-limiter) that prevents re-accepting this channel for a little bit.

If someone really wants to do something and keep a log of deleted channels, they can totally do that, and again the nodes that screwed up their channel are now hosed, that's their problem, but the spec doesn't need to mandate this.

[mtfs1]

Is there a future on using ZKP on pubkey ownership inside a large set as a way to eliminate the necessity on the UTXO announcement as spam filtering?

https://github.com/AdamISZ/aut-ct

This guy presented his project on delving bitcoin some months ago, but I have seen few people commenting on it.

I'm not a bitcoin dev, but I'm commenting here because I got really curious on this proposal, and had some fear that people could be missing something important.