Win32-OpenSSH
Win32-OpenSSH copied to clipboard
PowerShell
SSH session does not correctly release file handles after closing
Prerequisites
- [X] Write a descriptive title.
- [X] Make sure you are able to repro it on the latest version
- [X] Search the existing issues.
Steps to reproduce
After establishing a connection, my SSH client reuses this connection and creates a session every second over this connection, sending an exec command on ssh channel to execute some simple script commands, and then closes this session. After my client program ran continuously for about 20 days, my Windows machine hung due to exhausted memory. Analysis revealed that the sshd process had a large number of handles that were not correctly released, as well as a significant number of residual cmd.exe processes.
I tried upgrading to version v9.2.2.0p1-Beta, but the issue still persists. However, after switching to the Bitvise SSH Server software, this problem no longer occurred. As a comparison, the same logic works as expected when running on Linux and macOS servers. I believe this should be a relatively easy case to reproduce, so I won't go into further details.
Expected behavior
no leaks
Actual behavior
file handlers leak
Error details
No response
Environment data
Name Value
---- -----
PSVersion 5.1.22621.1778
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.22621.1778
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Version
8.6p1
Visuals
too many file handlers hold by sshd.exe
about 2000000 cmd.exe processes, all 32K page table used
@CJey can you provide repro steps?
@maertendMSFT ok，look the full client demo code
golang code
package main
import (
"fmt"
"golang.org/x/crypto/ssh"
)
func main() {
var user = "administrator"
var pass = "your password"
var host = "10.1.1.1:22" # your windows sshd entry address
var cfg = &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{
ssh.Password(pass),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}
var cli, err = ssh.Dial("tcp", host, cfg)
if err != nil {
panic(fmt.Errorf("Dial ssh failed, %w", err))
}
defer cli.Close()
fmt.Printf("Infinite looping, session open & session close\n")
for {
session, err := cli.NewSession()
if err != nil {
panic(fmt.Errorf("New ssh session failed, %w", err))
}
session.Run("echo leak")
session.Close()
}
}
execute this program，then watch the handlers column of sshd.exe from task manager，you will see the numbers keep growing
We are experiencing the same issues as well, we notice handles are increasing while the SSH connection is open and not released until the connection is closed, causing a memory leak. @maertendMSFT FYI
How can one reproduce this from the command line quickly?
I tried (from a Linux client):
$ ssh -N -M -o ControlPath=~/fooctl windows-hostname.org &
$ while ssh -o ControlPath=~/fooctl windows-hostname.org tasklist | grep sshd ; do sleep 0 ; done
sshd.exe 4592 Services 0 7,600 K
sshd.exe 8376 Services 0 10,464 K
sshd.exe 4868 Services 0 8,464 K
[...seconds later...]
sshd.exe 4592 Services 0 7,600 K
sshd.exe 8376 Services 0 10,480 K
sshd.exe 4868 Services 0 8,460 K
[...after returning from making a cup of tea...]
sshd.exe 4592 Services 0 7,600 K
sshd.exe 8376 Services 0 10,444 K
sshd.exe 4868 Services 0 8,784 K
[...]
The final column is “Mem Usage” output by tasklist.exe. No spectacular increase ...
@stratfoa Can you share the section of the curl script that opens and closes ssh sessions? That would help us with investigation
