SSHD client causing memory leak due to SSHD calls in vast number around 1.8k

[sumit1511]

Version

2.9.2

Bug description

We are using Apache mina jars v2.9.2 in our application. Our customer on this version, reported the memory leak issue with the Apache mina sftp client, and there is a build of 1.8k plus threads of sshd-SschClient.

Actual behavior

Getting memory leak with apache mina sftp client

Expected behavior

it should be able to process without memory leak.

Relevant log output

From the thread dump

Thread Name 	sshd-SshClient[fd98cb5e]-timer-thread-1
State 	Runnable
Java Stack
at sun/misc/Unsafe.park(Native Method) 
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:226(Compiled Code)) 
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2089(Compiled Code)) 
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1104(Compiled Code)) 
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:820(Compiled Code)) 
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1085(Compiled Code)) 
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145(Compiled Code)) 
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code)) 
at java/lang/Thread.run(Thread.java:826(Compiled Code)) 
Native Stack
ZwWaitForSingleObject+0x14 (0x00007FFAC378FC74 [ntdll+0x9fc74]) 
WaitForSingleObjectEx+0x93 (0x00007FFAC06A4513 [KERNELBASE+0x34513]) 
monitor_wait_original+0x917 (omrthread.c:4686, 0x00007FFAA3DE7277 [J9THR29+0x7277]) 
omrthread_monitor_wait+0x43 (omrthread.c:4402, 0x00007FFAA3DE2B13 [J9THR29+0x2b13]) 
internalAcquireVMAccessNoMutexWithMask+0x2fc (vmaccess.cpp:386, 0x00007FFAA0EEA16C [j9vm29+0x6a16c]) 
internalAcquireVMAccessClearStatus+0x98 (vmaccess.cpp:338, 0x00007FFAA0EE9CD8 [j9vm29+0x69cd8]) 
threadParkImpl+0x19e (threadpark.c:98, 0x00007FFAA0F4594E [j9vm29+0xc594e]) 
(0x00007FFA8EE5D2CE) 
(0x0000000000000004) 
(0x0000000000000001) 
(0x000000003B1B4A28) 
(0x00000001FFC5FB70)

Other information

We have checked other close issue https://github.com/apache/mina-sshd/issues/354 , suggestion was that use 2.9.2 version of jar so issue should be fix but still facinge same issue with 2.9.2

[tomaswolf]

The answer is still the same: normally there is one such thread per SshClient. If these tasks accumulate, check that you close your clients properly.

Is the code of that application publicly available somewhere? Then I might take look. Otherwise provide minimal code that reproduces the issue.

[roma2341]

I'm using spring-integration-sftp and I had thousand of "sshd-SshClient" threads because they are never being closed.

[tomaswolf]

Did you report a bug at spring-integration?

[roma2341]

@tomaswolf not yet, maybe i do something wrongly, i wrote about this because spring-integration-sftp uses MINA under the hood

[roma2341]

it seems that the issue was that I didn't call DefaultSftpSessionFactory.destroy() in spring-integration-sftp, but I will check if it helped later Update: Yes, it was because I had to call DefaultSftpSessionFactory.destroy().