bolts
bolts copied to clipboard
lightning
BOLT 4: Remove legacy format, make var_onion_optin compulsory.
My measurements a few weeks ago reveal that only 5 nodes do not advertize this feature, of over 17000. I have a patch to remove support from c-lightning, too.
Actually while updating eclair, I realized that we should also update the test vectors:
- remove https://github.com/lightning/bolts/blob/master/bolt04/onion-test-v0.json
- udpate https://github.com/lightning/bolts/blob/master/bolt04/onion-test-multi-frame.json to only contain variable-size payloads
That does mean a bit more work to do (sorry!).
Hmm, I'm out of time to update the multiframe test, though you're right :(
Concept ACK, we'll likely start with just making our existing feature bit required, as we have some existing tests that still assume this is relevant.
I'm looking at our node's statistics, and a non-negligible portion (~30%) of the payments we relay are still using the legacy format. I'm afraid most nodes advertise the feature, but without actually using it by default when sending payments. Can you check relay statistics on your nodes to see if it matches what I'm seeing? Are all implementations always generating onion with the new format (and since which version)?
Are all implementations always generating onion with the new format (and since which version)?
lnd has been generating with the new format as of version v0.8.0 (released October 2019). In our path finding code, we'll use the old/new depending on which feature bits the node advertises (so we support mixing old/new along a route).
In our path finding code, we'll use the old/new depending on which feature bits the node advertises (so we support mixing old/new along a route).
In that case that's probably not the culprit since more than 99% of nodes (including ours) advertise var_onion_optin.
I'm suspecting a popular wallet (or a few of them) is still using the legacy encoding for their payments...
on c-lightning IRC we have a crash report due to the old onion format too!
my node seems to be crashing due to invalid onion messages Unexpected message WIRE_OBS2_ONION_MESSAGE: using master at commit 7abc491f4c214f2bcf9e8f2b9066ca05dee0342b
on c-lightning IRC we have a crash report due to the old onion format too!
I find c-lightning should definitely not be crashing, even with the legacy encoding disabled it should simply fail the adds with a required_node_feature_missing or something similar, shouldn't it?
Some ppl on Twitter hypothesized that maybe a lot of the traffic is actually probing. I know in the past, some systems/applications would use an API for lnd that accept a raw routes for probing. I dug into things on our end, and realized that unless a user is setting some custom TLV record in the route (also not using mpp/amp), we'll default to using the legacy payload (there's a bool that defaults to false). So this might be contributing to that 30% number mentioned above...
We have a new major release coming up, so this could be an opportunity to change our API to default to the modern payload (can't thing of any downside really).
We've released version v0.14.3 of lnd which removes the ability for users to specify the legacy payload option using certain API calls. However if a node in the path needs the legacy format, then the normal path finding code will handle populating the correct payload.
It looks like lnd's release really helped, we're now only seeing ~5% of the payments we relay use the legacy format. Hopefully in a few months that will drop below 1% and we can deprecate it.
Hopefully in a few months that will drop below 1% and we can deprecate it.
Any good news here? :)
We're still seeing payments use the legacy onion format... Over the last 3 days, 2,8% of relayed payments used it. Over the last week, 1,6% of relayed payments used it. I'd still wait a bit before releasing a version that rejects those payments.
Payments using the legacy encoding have fallen below 0,2% of the total payments relayed on our node in the last week :tada: Let me know if other nodes see similar data, this is a good indication that we may be able to finally remove this soon!
Payments using the legacy encoding have fallen below 0,2% of the total payments relayed on our node in the last week tada Let me know if other nodes see similar data, this is a good indication that we may be able to finally remove this soon!
Confirmed: my node has seen no legacy since 17 June (over 12000 htlcs fwd since then), and the one before that was 2 July.
He's dead, Jim!
Squashed into a single commit, and included the test vector fixups. Will test them today...