heartbeat thread is sometimes not terminated upon client closing

[spew]

trafficstars

I am running into a frequent issue where the heartbeat thread does not get interrupted properly when close my SSHClient object. This happens often when the sshClient.connect is refused (I have a retry mechanism that retries this up to 20 times with fixed interval 10 second sleep). This leads to my application not exiting. Let me know if more information is needed -- I could attempt to write an isolated program to reproduce the issue.

If I had to guess it might be some sort of deadlock with the writeLock in TransportImpl as the Heartbeater needs to obtain that lock in its loop when it does a write(...)

[hierynomus]

For these kind of issues I would definitely welcome an isolated testcase which reproduces this behaviour. I you could supply one, I'm more than happy to fix this.

[hierynomus]

@spew I've changed a few things in the heartbeat mechanism for 0.11.0. Could you retry and see if this still happens, and else give me a small program that shows the problem?

Thx!

[spew]

Updating to 0.11.0 will report if/when we see this or close in a month or two if we don't.

[hierynomus]

Great, let me know!

[ksumit]

We hit this issue again, please see the stacktrace below:

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
    at net.schmizz.sshj.SSHClient.auth(SSHClient.java:217)
    at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:316)
    at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:365)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Timeout expired
    at net.schmizz.sshj.userauth.UserAuthException$1.chain(UserAuthException.java:33)
    at net.schmizz.sshj.userauth.UserAuthException$1.chain(UserAuthException.java:26)
    at net.schmizz.concurrent.Promise.retrieve(Promise.java:139)
    at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:69)
    at net.schmizz.sshj.SSHClient.auth(SSHClient.java:211)
    ... 15 more
Caused by: java.util.concurrent.TimeoutException: Timeout expired
    ... 18 more

[lukelast]

I've also seen this issue where I ended up with 100's of heartbeater threads that leaked. The threads would also prevent a lot of garbage from being collected.

I solved this problem by using a scheduled executor instead. I'm currently Integrating my changes of the heartbeat functionality to my fork and hopefully the will be accepted if it solves the problem.

[mdwn]

I'm seeing this as well when auth fails -- basically, trying to connect to a non-existent host with a public key. We've got a wrapper around sshj, but adding in a client.getConnection().getKeepAlive().interrupt() when trying to close the client seems to kill the threads here.

[mbrucco]

It happens when an host is unreachable with username and password too

[exceptionfactory]

This issue should be resolved with updates in PR #752.

[spew]

Awesome!