Simplify coin selection for sendcoins

[Chinwendu20]

Change Description

This PR fixes this issue: https://github.com/lightningnetwork/lnd/issues/6949#issue-1386947921 Depends on: https://github.com/btcsuite/btcwallet/pull/912 and https://github.com/lightningnetwork/lnd/pull/8729/

In this well detailed issue comment, the process of sending funds using utxos selected by the sender currently has over five steps in which one has to call various psbt APIs.

The goal of this PR is to improve user experience by including utxos to the sendcoins command and corresponding rpc structs (SendCoinsRequest and SendCoinsResponse ) which would enable the sender described to achieve the same aim with one command.

This is done by adding a new field to rpc structs SendCoinsRequest and SendCoinsResponse to enable users request for that functionality and enable the driving function (in which we modify its functionality as well) use these utxos when crafting the transaction for this functionality.

Additionally, the sweepall field in the above mentioned request and response rpc structs when true now not only sweeps ALL funds in the wallet but also ALL funds in the selected utxos when used in conjunction with the new select utxos field.

The lncli sendcoins command was updated to include the flag, utxo to enable this functionality on that end.

There were also slight internal logic change, where a slice of utxos are now accepted by relevant functions as variadic argument, functional options to enable this functionality.

Steps to Test

Steps for reviewers to follow to test the change.

Pull Request Checklist

Testing

• [ ] Your PR passes all CI checks.
• [ ] Tests covering the positive and negative (error paths) are included.
• [ ] Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

• [ ] The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
• [ ] Commits follow the Ideal Git Commit Structure.
• [ ] Any new logging statements use an appropriate subsystem and logging level.
• [ ] Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
• [ ] There is a change description in the release notes, or [skip ci] in the commit message for small changes.

📝 Please see our Contribution Guidelines for further guidance.

[coderabbitai[bot]]

[!IMPORTANT]

Review skipped

Auto reviews are limited to specific labels.

Labels to auto review (1)

• llm-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[Chinwendu20]

Hello @hieblmi here is the PR

[Chinwendu20]

~~I would be using the psbt API instead as this approach would not craft a transaction with all inputs selected by the user as pointed out here: https://github.com/btcsuite/btcwallet/pull/912#discussion_r1510907241~~

Modified it, no need for this

[Chinwendu20]

Hello @sputn1ck, do you think I should create a different function instead of adding an arg to an exported one as per your comment here: https://github.com/btcsuite/btcwallet/pull/912#discussion_r1510902312

[bennyhodl]

tACK.

Tested locally and worked as expected with one utxo, sweep, and multiple utxos. Would like to see this merged as I am signing PSBTs manually because my application requires 1 input --> 1 output transactions.

[bennyhodl]

Is there an update on this @Chinwendu20 ?

[Chinwendu20]

Is there an update on this @Chinwendu20 ?

Thanks I should push an update by the end of this week..

[Chinwendu20]

This PR is now dependent on the PR: https://github.com/lightningnetwork/lnd/pull/8729/

[Chinwendu20]

This PR is now dependent on https://github.com/lightningnetwork/lnd/pull/8750 and https://github.com/btcsuite/btcwallet/pull/928

[lightninglabs-deploy]

@sputn1ck: review reminder @hieblmi: review reminder

[hieblmi]

There are still itests failing.

[Chinwendu20]

I think the taproot test needs to create a new node and not use the general Alice node. Two tests can request sendCoins to sweep all coins at the same time, creating the same transaction same txid, the other test can get the txid first into the mempool making the sweepall sendcoins operation fail in the other. I do not know if that makes sense. If it does I can open a new PR for a fix.

In the meantime, this needs rebasing but I need to know where this stands in approval so I make changes at once.

[Chinwendu20]

Maybe all tests should create their own node to prevent cases like this?

[hieblmi]

I think the taproot test needs to create a new node and not use the general Alice node. Two tests can request sendCoins to sweep all coins at the same time, creating the same transaction same txid, the other test can get the txid first into the mempool making the sweepall sendcoins operation fail in the other. I do not know if that makes sense.

Not sure how this applies to the tests, in here lnd_taproot_test.go:128 sendcoins is called sequentially, why is the last call exceeding its deadline?

I think we should fix the itests in a separate commit in this PR.

[Chinwendu20]

When I checked the logs for Alice related to this test, I saw this

2024-05-24 09:38:40.442 [WRN] BTWL: Transaction 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d not accepted by mempool: txn-already-in-mempool
2024-05-24 09:38:40.592 [INF] LNWL: Inserting unconfirmed transaction 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d
2024-05-24 09:38:40.943 [DBG] LNWL: Marked address bcrt1pn388y03pjsedvrl2n0dxgyz7yup9q6snhyrqj67qh2aj8dq88udq6wfs78 used
2024-05-24 09:38:41.462 [INF] RPCS: [sendcoins] spend generated txid: 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d

sendcoins is called sequentially in that test but other tests using the Alice node would call sendcoins in parallel, right? So my initial thinking is that because the tx failed the TestMempool check and also that it would return an error while publishing the transaction it would time out with that error but the last log line shows that infact the the sendcoins rpc function exits successfully in spite of that and following the code it makes sense because ErrTxAlreadyInMempool is usually ignored.

Maybe if we rerun this it would pass, seems like a case of unprecedented slow server response

[Chinwendu20]

fyi I ran this test locally and it was successful.

[Chinwendu20]

@heebs the CI has been ran the failed tests that has to do with sendcoins are the ones using postgres backend, other itests pass. I think it is unrelated. Maybe as a separate issue we need to figure out why the response seems to be coming slower than expected for that case. Same things happened in this PR: https://github.com/lightningnetwork/lnd/actions/runs/9089660152/job/25069073966?pr=8759

[Chinwendu20]

2.) https://github.com/lightningnetwork/lnd/actions/runs/9270373663/job/25641222998?pr=8516#step:5:920 This I can repro locally with make itest-parallel backend=bitcoind dbbackend=postgres, but it runs a long time.

3.) https://github.com/lightningnetwork/lnd/actions/runs/9270373663/job/25641222017?pr=8516#step:5:924 This I can't repro, but it should get resolved if 2.) is resolved since it is the same error.

Responded to this here: https://github.com/lightningnetwork/lnd/pull/8516#issuecomment-2135505457

[hieblmi]

Ah right, I guess then it depends on if we can ignore the force-closure itest which is flaky and will be fixed in v0.18.1 according to @yyforyongyu

Could you confirm that this is the flake you are referring to? https://github.com/lightningnetwork/lnd/actions/runs/9270373663/job/25641223443?pr=8516#step:5:385

[Chinwendu20]

This doesn't look like a flake to me, will investigate - meanwhile could you rebase and push so we can rerun the CI.

I think it is unrelated to the PR, we can create a separate issue for that case so maybe we do not block this, same thing happens in another PR: https://github.com/lightningnetwork/lnd/actions/runs/9089660152/job/25069074366?pr=8759#step:5:767

We can track it here: https://github.com/lightningnetwork/lnd/issues/8806

[yyforyongyu]

Cool I think this is why the itests failed, so not related to this PR.

[Chinwendu20]

@guggero we have two approvals now, I think we can merge?

[hieblmi]

@guggero we have two approvals now, I think we can merge?

Wanted to make sure that @yyforyongyu's CRs were addressed, would be good to get his go too.

[Chinwendu20]

@yyforyongyu I think we do that already? I responded to that in the specific code piece you wrote the review in. Let me know if that works.

[lightninglabs-deploy]

@yyforyongyu: review reminder @chinwendu20, remember to re-request review from reviewers when ready

[guggero]

Replaced by https://github.com/lightningnetwork/lnd/pull/8955.