Add core-lightning as LN node provider.

[Reckless-Satoshi]

Is your feature request related to a problem? Please describe. We are currently relying on LND for RoboSats backend node. It does work pretty well. There is only a few things that might be critical: channel.db is hard to manage and performance/reliability could still be better (I guess?).

It would be great to implement core-lightning as backend, not as a replacement, but for exploration and extra redundancy. The higher performance and use of Postgres as database would be much appreciated. While it is not clear yet whether there will be some shenanigan or unexpected issue, the redundancy of having two node providers will always be of great value.

If core-lightning is available as node provider, future effort to improve RoboSats availability could be put into creating a "LN load balancer" that will smartly direct invoice creation / payments to a pool of nodes of different providers.

Describe the solution you'd like

We need a new api/lightning/node.py (LNnode provider) that can perform all of the same functions using core-lightning as backend instead of LND.

• [ ] Prerequisites not met By @louneskmt we need a hodl invoice plugin in Core-Lightning. Out of scope for RoboSats.
• [ ] The first step would be to add a core-lightning container running testnet3 to our dev docker-compose orchestration.
• [ ] It might be useful to implement unit tests test.node.py to learn about the current set up. The unit tests will be very helpful in the future, as they should receive exactly the same behavior regardless of whether the call was attended by LND or core-lightning in any future release of these.
• [ ] The bigger chunk of the work: write all core-lightning calls for every function.

I am unsure how big this task really is. But it certainly requires some very specific knowledge on interacting with Lightning node APIs.

Additional context This task is ⚡rewarded with 2M Sats⚡ . Drop a comment if you want to be assigned.

[louneskmt]

Hey! I can try to take care of this task.

[Reckless-Satoshi]

Hey! I can try to take care of this task.

Assigned! Good luck, this is a challenging task :) Reach us via @robosats in Telegram if you need help setting up the dev stack.

[louneskmt]

Would it be okay to use the Unix Domain Socket + the official Core Lightning python client to communicate with the node? I initially planned to use their new gRPC plugin to avoid the use of a new library, but it is still missing useful methods (like for decoding pay requests). The main python service would need access to the file socket (path).

[Reckless-Satoshi]

Sounds good to me. Have no experience with it, but I guess we can share a file socket into several containers (c-lightning and robosats python services) using a volume mount.

[louneskmt]

We can definitely do that with Docker, however it won't work on macOS (only on Linux). It is a known "won't solve" Docker issue. I learnt it the hard way a couple of months ago, after several hours of debugging lol

https://github.com/docker/for-mac/issues/483#issuecomment-758836836

[Reckless-Satoshi]

however it won't work on macOS (only on Linux)

Sad, it means a chunk of the stack won't work on the machines of many potential robo-developers 🤔 Are you in macOS?

Not an issue for RoboSats operation as it mostly relies on ubuntu server.

[louneskmt]

Yeah... Alternatively, I can look for all the missing gRPC methods and implement them on the plugin on Core Lightning. Maybe there are not that much (I gotta admit I've just checked if decodepay was there). They are missing anyway. We would then need waiting for a new Core Lightning release in order to use my changes with Robosats.

[Reckless-Satoshi]

Yeah... Alternatively, I can look for all the missing gRPC methods and implement them on the plugin on Core Lightning.

Maybe this missing gRPC method (decodepay) can be substituted by the json-RPC one? In any case, all the creative freedom to you :) If you think the plugin is a better solution, let's do it.

How hard would it be for us to build our own temporary Core Lightning image? There is no rush for this robosats feature, it will certainly take a lot of testing until we are comfortable to use it in the wild with real users.

[leshik]

You might want to use socat as a workaround to expose the UNIX domain socket over TCP.

[louneskmt]

AFAIU Core Lightning doesn't natively support hold invoices creation. However, it does provide a plugin hook invoice_payment that notifies when a payment is coming in before actually settling the HTLC. It is possible to use that to retain the HTLC settlement, and thus add support for hodl invoices. I couldn't find an existing plugin adding hodl invoices creation support using this.

We would need to implement such plugin, but I'm not sure yet of the additional amount of work needed.

[Reckless-Satoshi]

Haven't played with c-lightning myself. I see the CreateinvoiceRequest method asks for the Preimage param, but it probably retains the preimage and uses it to autosettle invoices on arrival. Also the method to query invoice status seem to only be able to return EXPIRED, PAID, UNPAID. We need the equivalent of ACCEPTED (LOCKED). We would also benefit from having CANCELLED, but not totally needed.

No experience with Core Lightning plugin, but sounds like the challenging task keeps getting harder :sweat_smile:

[louneskmt]

Yeah, looks like a full "hodl invoice" plugin needs to be written for Core Lightning. This is gonna be a bit too complex for me, so unfortunately I'm gonna have to drop this issue.

Happy to take care of the Robosats part if someone writes the plugin tho!

[Reckless-Satoshi]

I think that building this plugin is a bit far from the scope of RoboSats.

Not familiar with Core-Lightning development, are plugins developed in the main repository? It seems so. I have found a couple explicit mentions to "hold invoice" in the tests (link) and an old thread on the implementation of htlc_accepted hook(https://github.com/ElementsProject/lightning/pull/2267). Also found online explicit mentions of how Core Lightning plugins would allow new capabilities such as 'hodl invoices' back in 2019. However there does not seem to be such plugin or anyone working on it (at least I could not find).

Is the best way to proceed to simply open an issue and hope it eventually gets done?

Happy to take care of the Robosats part if someone writes the plugin tho!

I am un-assigning you and moving this task to a different category ("prerequisites not met"). Let me tip you with a bit of positive karma (40K Sats) for the valuable research you have done. Please share an invoice with long expiration time from a non-sensitive node_id :)

[daywalker90]

I'm currently developing a hold-invoice plugin for cln: https://github.com/daywalker90/hodlvoice related thread: https://community.corelightning.org/c/developers/hold-invoice-plugin and i could use some feedback in general and specific for this project. The plugin can create an invoice it will hold and has commands to accept or reject the invoice.

[Reckless-Satoshi]

Hey @daywalker90 fantastic news!

I believe LND's hodlinvoice API implementation is both elegant and simple. This plugin will probably serve best if it is as close as possible as a drop-in replacement for LND (where most HoldInvoices take place at the moment, so diversifying into CLN will be an easy transition)

In short, we need to AddHoldInvoice that has a user-defined preimage/r_hash. Then, for every invoice state change, we want to get notified via some QueryInvoiceState, and at will, we might want to act on this invoice by calling SettleInvoice or CancelInvoice.

Those 4 requests types would correspond to LND's:

• AddHoldInvoice https://lightning.engineering/api-docs/api/lnd/invoices/add-hold-invoice
• LookupInvoice https://lightning.engineering/api-docs/api/lnd/invoices/lookup-invoice-v2
• SettleInvoice https://lightning.engineering/api-docs/api/lnd/invoices/settle-invoice
• CancelInvoice https://lightning.engineering/api-docs/api/lnd/invoices/cancel-invoice

We can AddHoldInvoice with a given r_hash and a cltv_expirty (the user must have first created a preimage and hash it. The user is in charge of storing this preimage somewhere to be used later!). It is important that the CLTV expiry is not hardcoded by the plugin. RoboSats uses a few different ones trying to keep them as short as possible while safe. They are different for maker bond, taker bond and escrow according to the timers the users select with some margin for possible mining network speed ups / slow downs. In "human time" they vary from ~24h to 4 days (a maker bond can last as long as: 24h public + 8h deposit timer + 24h chat + safety margin for mining speed ups).

# Simplified draft
preimage = hashlib.sha256(secrets.token_bytes(nbytes=32)).digest()
r_hash = hashlib.sha256(preimage).digest()
memo= "This is a hold invoice"
value=num_satoshis
expiry= 600 # Invoice expiry seconds
cltv_expiry=cltv_expiry_blocks # HTLC expiry, in timechain blocks into the future

response = AddHoldInvoice(expiry=expiry, value=value, hash=r_hash, cltv_expiry=cltv_expiry, memo=memo...)

Then QueryInvoiceState / LookupInvoice that will return whether the Invoice is Open (nothing happened yet), Accepted (incoming HTLC received!), Cancelled (invoice was cancelled) or Settled (Invoice Settled). If the invoice is in the Accepted state, we should be able to call SettleInvoice(preimage) to close that payment settling the balance locally or, CancelInvoice(r_hash) to close it and let the balance stay at its origin.

You can take a look at how these are used by the RoboSats coordinator (click on them to see the full functions): https://github.com/Reckless-Satoshi/robosats/blob/8c66a5db829d1430614b07dbc35e0289a16798f5/api/lightning/node.py#L192 https://github.com/Reckless-Satoshi/robosats/blob/8c66a5db829d1430614b07dbc35e0289a16798f5/api/lightning/node.py#L239 https://github.com/Reckless-Satoshi/robosats/blob/8c66a5db829d1430614b07dbc35e0289a16798f5/api/lightning/node.py#L182 https://github.com/Reckless-Satoshi/robosats/blob/8c66a5db829d1430614b07dbc35e0289a16798f5/api/lightning/node.py#L172

(I would probably improve with respect to LND the fact that successfully Settling or Cancelling an invoice has an empty response)