Channel forcibly closed and funds not returned

[gustavocoding]

Funding tx was 731c6e69338f4cc58b595227cf5f2b8bf631e645a197445bcc7bf4f57f5098fb, I forcibly closed the channel id fb98507ff5f47bcc5b4497a145e631f68b2b5fcf2752598bc54c8f33696e1c73 and the state in the wallet is still CLOSING.

More than 144 blocks have passed since the funding transaction, shouldn't the funds have been returned?

[gustavocoding]

This is the stuck channel.

[gustavocoding]

Out of curiosity, how would I troubleshoot the issue? Any logfile in the android wallet? I could not find any info about this channel on any public lightning explorers such as http://1ml.com

[CaveRock]

Have a similar issue, have a channel whose creation txid does not exist on the network (can't be found by explorers). Now when trying to close the channel it has been sitting in the closing state for a while.

The problem now is not being able to re-open a channel to the same node due to an error message saying that duplicate channels are not allowed (understandable)

[pm47]

@gustavonalle this looks related to https://github.com/ACINQ/eclair/issues/546

@CaveRock that's a different issue, but I see you have opened a separate ticket

[andronoob]

Have a similar issue, too. The fund might be locked for 144 blocks, I'm waiting for the result. BTW, I'm connecting Eclair v0.2-beta1 running on my PC thru LAN.

[remyers]

I think I have a similar issue. I believe my repo steps were something like this:

1. open channel (then walked away from the phone)
2. when I came back, the channel was not opened and I think it timed out due to low tx fee
3. performed a forced close of the channel to recover channel funds
4. stuck in 'CLOSING (uncooperative)' state
5. I also cannot open a new channel with this node.

This is the funding tx

I've put some relevant screen shots below. It's the April 5 @ 14:59:16 transaction from the history:

Which does not show any refund of 4eea8... .

If I drill down on the opening transaction, it looks like this:

Here's what it looks like in the channels list:

And drilling down looks like this:

PS> Great App! I love the UI and ability to scan in QR codes to link channels, which works great with 1ml.com.

[gustavocoding]

@pm47 Can I see the logs in the android wallet to see why the channel is stuck on closing? Anything I can do to forcibly close a forcibly closing channel? 😀

[gustavocoding]

I was able find public info about the channel:

https://www.robtex.com/lightning/channel/568071478635462656

The closing tx has 160+ confirmations and one output to a bech32 address.

Does that mean the funds went to the other party? This would be really weird as I opened the channel, the other party went OFFLINE and I manually closed the channel.

[pm47]

Can I see the logs in the android wallet to see why the channel is stuck on closing?

@gustavonalle using android studio you will see the logs

[gustavocoding]

Does Android studio connect to my phone to extract the logs? Do I need a rooted phone?

[JukeboxRhino]

I also just experienced this issue. I'll post as much information as I can.

Channel Identifier: 10c3ad51ddc447d4a82ededf6b474e457519c2cd86936ee3618952981d7a9f9d Remote Node: 030312a32a1d38d27312c9596de2f9078b1b3bba522f7d204049dd5282b950762f Funding Transaction: 9d9f7a1d98528961e36e9386cdc21975454e476bdfde2ea8d447c4dd51adc310 Closing Transaction: 7006f41d24e16bc5a80a5f46983666143f88fc7d6c00a73cc9b8226ddfb0f9f7

This channel was opened between my phone wallet (eclair) and my desktop lightning node (desktop eclair). I'm fairly certain the app was running in the background on my phone when this happened. I have no reason to believe that my desktop node was offline, so there was no reason for a force close.

As of writing this the closing transaction is still unconfirmed. The channel has disappeared completely from my desktop node (no force closing message) and also from 1ml.com.

[gustavocoding]

@gustavonalle this looks related to ACINQ/eclair#546

#546 looks like the closing tx was not done due to wrong fees, whilst in my case it is in the blockchain but the wallet cannot recover from the closing state

[pm47]

Closing involves not only the commitment tx, but also all the txes that may send funds back to your wallet. Those txes might have the fee issue.

Le ven. 6 avr. 2018 à 16:37, Gustavo Fernandes [email protected] a écrit :

@gustavonalle https://github.com/gustavonalle this looks related to ACINQ/eclair#546 https://github.com/ACINQ/eclair/issues/546

#546 looks like the closing tx was not done due tonwrong fees, whilst in my case it is in the blockchain but the wallet cannot recover from the closing state

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ACINQ/eclair-wallet/issues/17#issuecomment-379273224, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7yvvYyz7B-qSBR8bhRIumG8AFEh_enks5tl32ggaJpZM4TJS16 .

[sstone]

@gustavonalle you don't need to root the phone (and you should not root a phone if you have a bitcoin wallet on mainnet) but you need to enable debugging (see https://developer.android.com/studio/debug/dev-options.html). then when you connect you phone to your PC for the first time it will ask you it you want to authorize USB debugging and wil show you the PC's fingerprint. Then you will be able to see the logs in the logcat tab of Android Studio

[gustavocoding]

@sstone Thanks. Will get the logs as soon as leave this beach and have access to a computer 😁

[sstone]

@JukeboxRhino I'm not sure I understand your problem: the closing tx that you post is confirmed and has been spent ?

[JukeboxRhino]

@sstone I think it's because of the issue with eclair desktop linked above (https://github.com/ACINQ/eclair/issues/546). If my desktop node was the one to force close the channel, then my phone wallet would have been able to spend the coins, which it was.

[sstone]

@remyers your closing tx needs to be confirmed 144 times before it can be spent, it was not when you posted but now the phone should be able to spend the closing tx and return the funds to your bitcoin wallet (you should see a new incoming transaction). What happens when you start eclair ?

[sstone]

@JukeboxRhino sorry I still don't understand. the funding tx is spend by a closing tx with 2 outputs, one that can be spent immediately (I guess it was your desktop output, it was spent and the channel closed), the other that cannot be spent before it has been confirmed 144 times and that's not the case yet

[gustavocoding]

@sstone @pm47 I attached the phone to Android Studio but the logs are just android level debug statements regarding UI and events which are pretty useless.

04-07 21:30:49.680 3726-6161/? I/MultiWindowManagerService: setTaskDimensions: Task=TaskRecord{85666b5d0 #1038 A=fr.acinq.eclair.wallet.mainnet U=0 StackId=1 sz=1} minWidth=-1 minHeight=-1 maxWidth=-1 maxHeight=-1
04-07 21:30:50.270 6341-6341/? I/SamsungIME: [IMI] onStartInput - caller packageName : fr.acinq.eclair.wallet.mainnet
04-07 21:34:18.301 3726-3825/? I/WindowManager_SurfaceController: Destroying surface Surface(name=fr.acinq.eclair.wallet.mainnet/fr.acinq.eclair.wallet.activities.ChannelDetailsActivity_dim) called by com.android.server.wm.WindowStateAnimator.finishExit:587 com.android.server.wm.AppWindowAnimator.stepAnimationLocked:493 com.android.server.wm.WindowAnimator.updateAppWindowsLocked:227 com.android.server.wm.WindowAnimator.animateLocked:919 com.android.server.wm.WindowAnimator.-wrap0:-1 com.android.server.wm.WindowAnimator$1.doFrame:180 android.view.Choreographer$CallbackRecord.run:925 android.view.Choreographer.doCallbacks:702 

Furthermore, it does not allow to show logs from when I closed the channel to see why it didn't close properly.

When I asked about logging, I was asking about wallet logs, to try to understand why the channel is permanently on CLOSING state.

Does the wallet store logs somewhere? Only by looking at 2 or 3 days ago logs it'd be possible to understand why the wallet failed to refund the channel

[gustavocoding]

I reproduced the issue with the emulator, opening a 1mBTC channel (mainnet) with 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b@167.99.50.31:9735 and closing it forcibly on my side. No refund on-chain tx can be seem, and the logs only printed:

04-08 10:34:41.470 3317-19435/fr.acinq.eclair.wallet.mainnet D/EclairEventService: Channel 18db723e56626a23ac05953a3901a9a0e26f3afe3f2316b3af820d9738daefc6 changed state to CLOSING

There is a suspect logging at the start though, when initialising the actor system's loggers:

04-08 10:49:46.102 27028-27028/fr.acinq.eclair.wallet.mainnet I/System.out: [ERROR] [04/08/2018 10:49:46.050] [main] [EventStream(akka://system)] unknown akka.loglevel WARN
    akka.event.Logging$LoggerException: 
        at akka.event.LoggingBus$$anonfun$2.apply(Logging.scala:100)
        at akka.event.LoggingBus$$anonfun$2.apply(Logging.scala:98)
        at scala.Option.getOrElse(Option.scala:121)
        at akka.event.LoggingBus$class.startDefaultLoggers(Logging.scala:98)
        at akka.event.EventStream.startDefaultLoggers(EventStream.scala:26)
        at akka.actor.LocalActorRefProvider.init(ActorRefProvider.scala:623)
        at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:620)
        at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:617)
        at akka.actor.ActorSystemImpl._start(ActorSystem.scala:617)
        at akka.actor.ActorSystemImpl.start(ActorSystem.scala:634)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:109)
        at akka.actor.ActorSystem.apply(Unknown Source:2)
        at fr.acinq.eclair.wallet.App.<init>(App.java:74)

I understand that the eclair Scala core is responsible for all 'heavylifting' of the wallet, maybe the ERROR above is causing the logs to be missed?

[pm47]

I reproduced the issue with the emulator, opening a 1mBTC channel (mainnet) with 02ad6fb8d693dc1e4569bcedefadf5f72a931ae027dc0f0c544b34c1c6f3b9a02b@167.99.50.31:9735 and closing it forcibly on my side. No refund on-chain tx can be seem

The refund transaction can't be published before the commitment tx reaches 144 blocks, which isn't the case yet for your tx I think. This is what makes the bug difficult to diagnose

I understand that the eclair Scala core is responsible for all 'heavylifting' of the wallet, maybe the ERROR above is causing the logs to be missed?

Interesting, we'll have a look at that

[gustavocoding]

The refund transaction can't be published before the commitment tx reaches 144 blocks, which isn't the case yet for your tx I think. This is what makes the bug difficult to diagnose

Yeah, I though about that, I will let the emulator running for a while then.

[gustavocoding]

Btw, is the commitment tx visible in the logs or in the wallet? I can see recurring events printed:

04-08 11:32:22.256 27028-28246/fr.acinq.eclair.wallet.mainnet D/PaymentSupervisor: Received TransactionConfidenceChanged message: TransactionConfidenceChanged(5f42b974e548ff65d8753f483afd79a1c29e9196230f3359d828293134445654,14)
04-08 11:32:22.285 27028-28246/fr.acinq.eclair.wallet.mainnet D/PaymentSupervisor: Received TransactionConfidenceChanged message: TransactionConfidenceChanged(c6efda38970d82afb316233ffe3a6fe2a0a901393a9505ac236a62563e72db18,12)

But those 2 tx are for the initial on-chain tx in my wallet and for a cooperative close I've done

[pm47]

Those are for transactions related to the on-chain wallet, which is separated from the lightning part. If the refund transaction was sent, it would appear in these messages

[gustavocoding]

@pm47 The log error is fixed by https://github.com/ACINQ/eclair-wallet/pull/28. I will change it to DEBUG locally to see if I can catch any error with the force closing of channels

[irekzielinski]

Seem to have a similar issue. Opened channel to you guys (Acinq node) - but I think channel was depleted so all payments were failing. Tried to close it cooperatively, but this was not working neither. After trying force close - channel seems to be stuck at CLOSING for few days now.

Happy to help if needed. ps. looks like balance went to this address: bc1qzgkcstfef869qhd08n0vl7ewca994w377xunw3p93ts4dg5wwnmsm2faqz

[pm47]

We have identified the bug and are working on a fix. There is a chance we may be able to recover funds without updating the app.

In the meantime please 🚨 DO NOT DELETE YOUR APP DATA 🚨

cc @robtex @j3bm3ch @Mindphreaker @ken2812221 @griswaalt @dognip

[sek3]

In facing the same issue here. I opened a channel and then realized I hadn't enough funds for the LN transaction I wanted to do. I force-closed it 10 min after opening the channel, so I believe the channel wasn't fully confirmed yet. This was on Friday. In yet to receive the funds back after 400+ confirmations.

[pm47]

We may have found a way to retrieve funds without updating the app. If you have encountered the bug please ping me on our gitter thanks!