WIP: Trampoline forwarding

[arik-so]

Forward Trampoline onions. Currently a work in progress, still missing these cleanups:

• [x] New HTLCSource variant for storing the session_priv instead of throwing them away
• [x] Tests for forwarding failures
• [x] Additional tests for failures that can occur both when forwarding and receiving Trampoline onions
• [x] Determine skimmable amount, charging a routing fee

[ldk-reviews-bot]

👋 Thanks for assigning @joostjager as a reviewer! I'll wait for their review and will help manage the review process. Once they submit their review, I'll check if a second reviewer would be helpful.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 88.63965% with 157 lines in your changes missing coverage. Please review.

Project coverage is 89.17%. Comparing base (481e5c7) to head (688ad3f).

Files with missing lines	Patch %	Lines
lightning/src/ln/channelmanager.rs	70.92%	111 Missing and 5 partials :warning:
lightning/src/ln/onion_payment.rs	57.53%	25 Missing and 6 partials :warning:
lightning/src/ln/blinded_payment_tests.rs	99.20%	7 Missing :warning:
lightning/src/chain/channelmonitor.rs	33.33%	2 Missing :warning:
lightning/src/ln/onion_utils.rs	95.45%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3711      +/-   ##
==========================================
+ Coverage   89.13%   89.17%   +0.04%     
==========================================
  Files         156      156              
  Lines      123477   124824    +1347     
  Branches   123477   124824    +1347     
==========================================
+ Hits       110056   111308    +1252     
- Misses      10752    10828      +76     
- Partials     2669     2688      +19     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[arik-so]

Hey guys, rephrased the commits, addressed what I think was most of your comments, and would like to squash a large portion of the fixup commits that ended up in here.

[valentinewallace]

Thinking about the current approach in the last commit where we find a route --

I think things may be cleaner if we move a lot of the handling of trampoline forwards to the outbound_payments module. In this approach, we would have a similar method to the existing send_payment_* methods there that creates route parameters, registers a PendingOutboundPayment::Retryable that's a trampoline, and forwards via the internal method find_route_and_send_payment.

The benefit of this approach is that retries will be handled automatically and it keeps us from adding a bunch of code to process_pending_htlc_forwards/adds more encapsulation in general.

I think some changes needed for this would be:

• when we're processing a trampoline forward in process_pending_htlc_forwards, we'll call into the outbound payments module to handle the pathfinding and the forward, and if it fails then fail the trampoline HTLC backwards
• modify ChannelManager's send_payment_along_path to construct the correct HTLC source and payment onion, since this method is called from outbound_payments
• if an outbound trampoline forward fails, when we notice this in fail_htlc_backwards_internal we'll call another new method PendingOutboundPayments::trampoline_htlc_failed(..) similar to the fail_htlc method in that module, which will either retry for us or tell us to fail the HTLC(s) backwards
• will need to make sure we generate the right events/don't generate PaymentFailed for failed trampolines

Thoughts on this alternate approach?

[arik-so]

That sounds very reasonable. I'm on it.