Stuck closing channel "Can't send commit: waiting for revoke_and_ack"

[owtyahyg6]

Issue and Steps to Reproduce

After issuing the following command in v0.24.02.1 to close a channel I have with an LND peer: ./bin/lightning-cli --conf etc/lightningd_2.conf close -k id=[*****peer_id****] unilateraltimeout=0 destination=[*****dest_address*****] feerange='[ "30000perkb", "60000perkb" ]' , the command was stuck in that state for a very long time. After half an hour or so I canceled it and I tried again, resulting in the command hanging again. Here is what I see in the logs:

DEBUG   [*****peer_id*****]-channeld-chan#4: pid 2428358, msgfd 77
DEBUG   [*****peer_id*****]-chan#4: Already have funding locked in
DEBUG   [*****peer_id*****]-channeld-chan#4: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1
DEBUG   [*****peer_id*****]-channeld-chan#4: init LOCAL: remote_per_commit = [*****hex_2*****], old_remote_per_commit = [*****hex_1*****] next_idx_local = 1 next_idx_remote = 2 revocations_received = 0 feerates { SENT_ADD_COMMIT:2006 SENT_ADD_ACK_REVOCATION:2001 } range 752-551730 blockheights { SENT_ADD_ACK_REVOCATION:0 }, our current [*****block_height*****]
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_out WIRE_CHANNEL_REESTABLISH
DEBUG   [*****peer_id*****]-channeld-chan#4: billboard: Sent reestablish, waiting for theirs
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_in WIRE_CHANNEL_REESTABLISH
DEBUG   [*****peer_id*****]-channeld-chan#4: Got reestablish commit=1 revoke=0 inflights: 0, active splices: 0
DEBUG   [*****peer_id*****]-channeld-chan#4: Splice resume check with local_next_funding: omitted, remote_next_funding: empty, inflights: 0
DEBUG   [*****peer_id*****]-channeld-chan#4: Retransmitting channel_ready for channel [*****channel_id*****]
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_out WIRE_CHANNEL_READY
DEBUG   [*****peer_id*****]-channeld-chan#4: next_revocation_number = 0
DEBUG   [*****peer_id*****]-channeld-chan#4: option_static_remotekey: fields are correct
DEBUG   [*****peer_id*****]-channeld-chan#4: Retransmitting commitment, feerate LOCAL=2001 REMOTE=2006, blockheight LOCAL=0 REMOTE=0
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_out WIRE_UPDATE_FEE
DEBUG   [*****peer_id*****]-channeld-chan#4: Creating commit_sig signature 1 [*****commit_sig*****] for tx [*****channel_tx*****] wscript [*****script*****] key [*****key*****]
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_out WIRE_COMMITMENT_SIGNED
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_out WIRE_SHUTDOWN
DEBUG   [*****peer_id*****]-channeld-chan#4: billboard: Channel ready for use. Shutdown messages exchanged.
DEBUG   [*****peer_id*****]-channeld-chan#4: No upgrade: we retransmitted
DEBUG   [*****peer_id*****]-chan#4: channel_gossip: reestablished
DEBUG   [*****peer_id*****]-channeld-chan#4: billboard perm: Reconnected, and reestablished.
DEBUG   [*****peer_id*****]-channeld-chan#4: billboard: Channel ready for use. Shutdown messages exchanged.
DEBUG   [*****peer_id*****]-channeld-chan#4: peer_in WIRE_SHUTDOWN
DEBUG   [*****peer_id*****]-channeld-chan#4: billboard: Channel ready for use. Shutdown messages exchanged.
DEBUG   [*****peer_id*****]-channeld-chan#4: Can't send commit: waiting for revoke_and_ack

If I just Ctrl-C and reissue the command, then the last line above is the only one that gets generate. Here is the status I see from listpeers:

"state_changes": [
                  {
                     "timestamp": "2024-04-13TXX:XX:XX.XXXXZ",
                     "old_state": "CHANNELD_AWAITING_LOCKIN",
                     "new_state": "CHANNELD_NORMAL",
                     "cause": "user",
                     "message": "Lockin complete"
                  },
                  {  
                     "timestamp": "2024-04-14TXX:XX:XX.XXXXZ",
                     "old_state": "CHANNELD_NORMAL",
                     "new_state": "CHANNELD_SHUTTING_DOWN",
                     "cause": "user",
                     "message": "User or plugin invoked close command"
                  }
               ],
               "status": [
                  "CHANNELD_SHUTTING_DOWN:Reconnected, and reestablished.",
                  "CHANNELD_SHUTTING_DOWN:Channel ready for use. Shutdown messages exchanged."
               ]

I don't see a closing tx id anywhere, there is nothing in the mempool for my close_to address either. What is going on and how can I unstuck this operation? Thanks!

[owtyahyg6]

The error is generated from channeld/channeld.c, due to the following condition returning true: peer->revocations_received != peer->next_index[REMOTE] - 1

The channel is still stuck in the same state three weeks later...