clnrest: replace with rust plugin

[daywalker90]

This is intended to be a drop-in replacement for @ShahanaFarooqui's clnrest.py. Python dependency management, especially in the context of CLN is not easy. Quite a few people struggled to setup CLN + python dependencies on modern unix OS. E.g. on debian 12 you have great choices like pip install --user --break-system-packages -r requirements.txt (you are only running CLN with this user right?) or hard-coding the PATH and PYTHON_PATH variables in your systemd service.

Installing a binary is easier, so @chrisguida bribed me into rewriting clnrest.py in rust.

In a future PR i intend to support creating dynamic REST routes for other plugins using @gudnuf PR's #7507 and #7508

[rustyrussell]

Nice! Might take a while to reach feature parity, I've asked @ShahanaFarooqui to review...

[daywalker90]

Thanks for the quick feedback @niftynei ! If you are new to rust i can recommend https://doc.rust-lang.org/book/

[daywalker90]

Can someone explain to me why the CI is regarded?

One test in CI:

 l1.daemon.logsearch_start = 0
>       assert l1.daemon.is_in_log(r"Error code 1501: Not authorized: Not derived from master")
E       AssertionError: assert None
E        +  where None = <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7fa9c910a8e0>>('Error code 1501: Not authorized: Not derived from master')
E        +    where <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7fa9c910a8e0>> = <pyln.testing.utils.LightningD object at 0x7fa9c910a8e0>.is_in_log
E        +      where <pyln.testing.utils.LightningD object at 0x7fa9c910a8e0> = <fixtures.LightningNode object at 0x7fa9c7766fa0>.daemon

throws this error while the logs clearly show it's there:

lightningd-1 2024-08-08T10:18:31.133Z INFO plugin-clnrest: Error code 1501: Not authorized: Not derived from master

I've never had this test fail locally.

[daywalker90]

Built ontop of #7544 after getting merged on master. Fixes compatibility with multi options.

[niftynei]

as a nixos user, i am very interested in this move away from python!

[chrisguida]

Feature request: log which runes were used for which calls. That way an organization can track which users authorized which transactions by giving different runes to different users.

[daywalker90]

Feature request: log which runes were used for which calls. That way an organization can track which users authorized which transactions by giving different runes to different users.

I've done it like this:

INFO    plugin-clnrest: Authorized `ysx3Jj5nOUWk3qujMHD8qo5Lcc0UlsXoDyXuhEC4_a49MA==` access to `invoice` with params: `{\"amount_msat\":1000,\"description\":\"owiegh\",\"label\":\"owigerjh\"}`

Also rebased on master and fixed tests for Valgrind (had to wait for stuff longer because Valgrind is so slow).

[chrisguida]

Awesome!!! Hmm, I'm realizing that these will be sensitive to have in the logs... is there any utility to clean stuff like this out?

Or maybe we can put in the db instead?

[chrisguida]

Oh actually you can reference the runes by their index instead of their actual value, that might be the way to go?

[daywalker90]

I've replaced the actual rune with the rune id (at the cost of an extra rpc call to showrunes) and formatted the string a bit better for people who want to parse it:

"Authorized rune_id:`{}` access to method:`{}` with params:`{}`"

[chrisguida]

This plugin has been added to https://github.com/lightningd/plugins#available-plugins if anyone wants to try it out!

See "clnrest-rs"

[chrisguida]

Hey @JosephGoulden would you mind taking a look at the CI failure on the nix flake for this PR? https://github.com/ElementsProject/lightning/actions/runs/13021608161/job/36324662522?pr=7509

For some reason DNS appears not to be working? What might cause this?

curl: (6) Could not resolve host: github.com

[chrisguida]

@JosephGoulden @ShahanaFarooqui I spoke with @daywalker90 and I think the problem should be solved by using the vendored version of the utoipa-swagger-ui dependency.

[JosephGoulden]

Hi. The flake checks run in a sandbox that doesn't allow internet access, everything has to be defined and downloaded beforehand. Looks like you've fixed it now anyway.

[ShahanaFarooqui]

@JosephGoulden @ShahanaFarooqui I spoke with @daywalker90 and I think the problem should be solved by using the vendored version of the utoipa-swagger-ui dependency.

Thanks, it worked :).

@daywalker90, thank you for the PR. This plugin is significantly faster and more reliable than its Python counterpart. While most of your Rust changes are intact, I updated the Makefile and Dockerfile after deciding to fully replace the Python plugin with the Rust version.