splice: Script for complex splices

[ddustin]

PR reviewers: It might be good to start here to learn the splice script syntax: https://github.com/ddustin/lightning/blob/ddustin/splice_script/doc/lightning-splice.7.md

New compiler for splice scripts that parses splice scripts, validates them, converts them to json and back again.

A test is included that puts some advanced and complex scripts into the parser and validates the results that come out.

Changelog-Added: Splice script parser — takes a custom splice query language to bundle multiple complex splices into a single task in a simple way.

[ddustin]

Mh What do you mean by "Scripts"? and why now we need a "compiler" for splicing? 😕

I feel that I am missing somethings?

Proper documentation is coming but here's a quick summary:

The script isn't required to be used -- there will also be a json option and, of course, the lower level api is available.

When doing splices across multiple channels it quickly becomes unwieldy for users. Using the script is a nice shorthand to make it easier -- and also allows you to do "dry runs" and confirm it's doing what you'd like.

Say you wanted to take 50% of funds and splice them into three other channels, in splice script it's quite simple:

84ff50 -> 50%;
* -> cecf35;
* -> 4fe310;
* -> c14f60;

While working on it, it made sense to add a bunch of other useful things like being able to specify amounts as whole numbers, percentages, or * (split amount among all *s); channel queries; onchain bitcoin addresses; and the onchain wallet.

By building this more powerful part first it becomes trivial to implement the simpler splice commands. For instance:

Splice out 10 million sats

84ff50 -> 10M;

Splice in 10 million sats

wallet -> 10M;
* -> 84ff50;

Cross splice 10 million sats

84ff50 -> 10M;
* -> cecf35;

Pay 10M sats onchain using channel funds

84ff50 -> 10M+fee;
10M -> 1JfbZRwdDHKZmuiZgYArJZhcuuzuw2HuMu;

[ddustin]

You can see a description of what it does here: https://github.com/ElementsProject/lightning/blob/ab42c231624664e33ee4437f2b3028b519aa8f53/doc/lightning-splice.7.md

[cdecker]

Ok, I'd like to have some more eyes on the script syntax, so I removed the milestone for now. Thanks for the excellent work @ddustin, and I'm sure we'll get some form of this merged by the next release :clap:

[chrisguida]

Anyone know what's up with this backslash in the docs for user_provided_sats? It's both in the markdown preview and in the manpage, but it appears to be named the right thing in the schema, so maybe something is just off in the docs rendering?

SYNOPSIS
       (WARNING: experimental-splicing only)

       splice [script] [dryrun] [json] [psbt] [user_provided_\sats]

[chrisguida]

I can't seem to get past this:

$ l1-cli splice '039751b -> 10M'
{
   "code": -32602,
   "message": "Splice script compile failed",
   "compiler_error": [
      "Missing channel or address equivilent. Each line must contain one",
      "039751b -> 10M",
      "           ^",
      "Compiler phase: make_segments"
   ]
}

What am I doing wrong?

[chrisguida]

Oh hmm I guess it wanted the channel_id not the peer_id.

Now it's just hanging...

[chrisguida]

If I do dryrun then it doesn't hang

$ l1-cli splice "c0a77695 -> 10M;" true
{
   "dryrun": [
      "c0a77695f94848c330f62bb1b7112ba2bf153e5f221c0155becc43299415f93d withdraw 10000000sat",
      "put 100% of rest into wallet"
   ]
}

[chrisguida]

Hmm now it says

$ l1-cli splice "c0a77695 -> 10000sat;"
{
   "code": 355,
   "message": "Currently waiting on previous splice command to finish."
}

I guess the old splice is still hanging somewhere?

[ddustin]

Thanks for reviewing!

SYNOPSIS
       (WARNING: experimental-splicing only)

       splice [script] [dryrun] [json] [psbt] [user_provided_\sats]

Ah it was a typo, nice spotting.

Oh hmm I guess it wanted the channel_id not the peer_id.

Now it's just hanging...

Yeah it needs a channel id or a channel query. If you want to use a node id you can specify "the first channel we have with node id" by saying 039751b:0 or 039751b:?

Might be worth making a clearer error message for this case 🤔

$ l1-cli splice "c0a77695 -> 10000sat;"
{
   "code": 355,
   "message": "Currently waiting on previous splice command to finish."
}

I guess the old splice is still hanging somewhere?

Yeah that happened because a previous splice got stuck prior or is waiting for user input (ie user signatures). Restarting the node will clear it.

If you remember which command made the splice get stuck that would be helpful 🙏

[chrisguida]

This was the one that hung

$ l1-cli splice "c0a77695 -> 10M;"
^C

Maybe because I didn't have that much in the channel? I just kinda slapped that in there without thinking much, guessing that's a splice out?

Edit: ah yeah, docs say "Splice out 10 million satoshis to the onchain wallet" xD

Might be worth it to return some kind of error message instead of hanging though?

[chrisguida]

Ok I restarted and tried it with a smaller amount and it seems to have worked!

$ l1-cli splice "c0a77695 -> 10000sat;"
{
   "psbt": "cHNidP8BAgQCAAAAAQQBAQEFAQIBBgEDAfsEAgAAAAABAP21AQIAAAAAAQJ1c2jX5HBTkzsEVasZnFd4tRbXb9QY+Ws1nnfQYjTVzwAAAAAA/f///4H8KHX0JJL0Na5sxWcu78ag+sW6MGl3Sajj9fBwgBQYAAAAAAD9////A/BZ9AUAAAAAIlEg5bV970yfonIJ0MpBPnJvpxi29LLnS3GLwK12WqUpMHkanuYFAAAAACJRIKZ9GbEKjvEgVWwI0hhGSOftkX7Q4lFrGyf+2PzsFHOd4MgQAAAAAAAiACDLZac0GecjZnnNTqJg5oadKounzGzU5eAM+uXGGQlgrQJHMEQCIFlmZJRHMig5YihgBwnuDvXGHn9PD9LzEjPDTWnlIxvbAiARUxLInnqYASliMbIQsvDpaTqkDgVJ2C1bBDzizu81kwEhA7iCzk9M8Npoyx2DEo38CkGsoGIJ46kA3gq+pnzgRS6sAkcwRAIgS10MidyLHnu0TV0meRGVWHsf7gwXvpEg+pktGbp0+/8CIBWKvQizYlnpcExtrJcpXdlpubRZFmBYwFC37X+HyVNfASEDLpZHCRF9B+n4HD7Je9QItJwNd5ub8kxh6mZN7ohZEEz0AAAAAQEr4MgQAAAAAAAiACDLZac0GecjZnnNTqJg5oadKounzGzU5eAM+uXGGQlgrSICAo6BpOai25V7gdMYxODX1ykKBhUPqHdSJ/YAAHr/V8DRRzBEAiBAQBWz4ulhD4k2TYdE/kDl7cs9+0emd95mx90Wob/hcwIgEePQiTkU0BD4s9mvKdc1JLZaUnKqnQzOI9z604m5GwABAQMEAQAAAAEOIPC7IXX1IS6zUoQvcjR6AVGJPDdrkI6UGT/oRdXQ4tURAQ8EAgAAAAEQBAAAAAAM/AlsaWdodG5pbmcBCLTD7bTesGowAAEDCDYmAAAAAAAAAQQiUSCLlzxf8KSQryCUGZhXUCSooXsoo1KCaIGjAFmj7JIspwz8CWxpZ2h0bmluZwEIQgenL51TwwQAAQMI0KEQAAAAAAABBCIAIMtlpzQZ5yNmec1OomDmhp0qi6fMbNTl4Az65cYZCWCtDPwJbGlnaHRuaW5nAQiZvH8MEdONxAA=",
   "txid": "4a282a73dc411412f27d4de02ab4fad87e398207164c49fb3a907599f3adc8f8"
}

[chrisguida]

Interesting, this is still here even though I mined 6 blocks and the channel is back to normal

         "inflight": [
            {
               "funding_txid": "4a282a73dc411412f27d4de02ab4fad87e398207164c49fb3a907599f3adc8f8",
               "funding_outnum": 1,
               "feerate": "253perkw",
               "total_funding_msat": 1090000000,
               "our_funding_msat": 1000000000,
               "splice_amount": -10000,
               "scratch_txid": "5a8909f2114723d754c9415a0a09b925487e293a5c972545751cc9a7da05cb52"
            }
         ],

How long does this output stick around?

[ddustin]

This was the one that hung

$ l1-cli splice "c0a77695 -> 10M;"
^C

Maybe because I didn't have that much in the channel? I just kinda slapped that in there without thinking much, guessing that's a splice out?

Edit: ah yeah, docs say "Splice out 10 million satoshis to the onchain wallet" xD

Might be worth it to return some kind of error message instead of hanging though?

Yeah it should definitely be throwing an error in this case.

Interesting, this is still here even though I mined 6 blocks and the channel is back to normal

         "inflight": [
            {
               "funding_txid": "4a282a73dc411412f27d4de02ab4fad87e398207164c49fb3a907599f3adc8f8",
               "funding_outnum": 1,
               "feerate": "253perkw",
               "total_funding_msat": 1090000000,
               "our_funding_msat": 1000000000,
               "splice_amount": -10000,
               "scratch_txid": "5a8909f2114723d754c9415a0a09b925487e293a5c972545751cc9a7da05cb52"
            }
         ],

How long does this output stick around?

It shouldn't be -- nice catch.

[ddustin]

$ l1-cli splice '039751b -> 10M'
{
   "code": -32602,
   "message": "Splice script compile failed",
   "compiler_error": [
      "Missing channel or address equivilent. Each line must contain one",
      "039751b -> 10M",
      "           ^",
      "Compiler phase: make_segments"
   ]
}

We could auto detect that it is a node id and fill in the channel id as long as there is only one channel with the node. I wonder if that's a good idea 🤔.

[chrisguida]

Yeah I guess I instinctively tried a peer id because I was used to this behavior:

If the given id is a peer ID (66 hex digits as a string), then it applies to the active channel of the direct peer corresponding to the given peer ID. If the given id is a channel ID (64 hex digits as a string, or the short channel ID blockheight:txindex:outindex form), then it applies to that channel.

[ddustin]

Fix the doc newline issue

[rustyrussell]

OK, can we make this experimental, or a dev- command? I'm not convinced this is the way we want to go, but it's great for exercising all the nasty cases, and may well be useful for larger nodes.

Needs rebase because it was now "added v24.11" not "v24.08", sorry :(

[ddustin]

OK, can we make this experimental, or a dev- command? I'm not convinced this is the way we want to go, but it's great for exercising all the nasty cases, and may well be useful for larger nodes.

Needs rebase because it was now "added v24.11" not "v24.08", sorry :(

I changed the command to "dev-splice," updated the added tag to 24.11, and fixed a handful of issues that came up from rebasing.

Can we push this one through once it passes CI @rustyrussell?

[rustyrussell]

Ack! Will apply once CI is happy!

[rustyrussell]

Accidentally told GH to update to latest master and it did a merge. Gah! Rebased instead.

[rustyrussell]

Now a proper rebase, taking into account jsonrpc request API changes. Also minor fixes since it didn't bisect build before.