vscode-remote-release icon indicating copy to clipboard operation
vscode-remote-release copied to clipboard

Zombie Process (<defunct>) and Lingering Processes in VS Code Servers

Open zakkg3 opened this issue 9 months ago • 1 comments

Is there an existing issue for this bug?

  • [x] I have searched the existing open issues and found none that apply.
  • [x] If I find any issue of interest that is related or closed, I will included a link to it in this issue.

Required Troubleshooting Steps

  • [x] I have read and performed the troubleshooting steps
  • [x] I have tried both values of the remote.SSH.useLocalServer setting
  • [x] My issue was not covered in the Tips and Tricks linked from the Troubleshooting Wiki.
  • [x] I will include a complete copy of my Remote - SSH logs by running Remote-SSH: Show Log in the command palette or from View > Output in the menu bar

[Optional] Diagnose with Copilot

I did not ask the @remote-ssh participant for help

In step 2 of the troubleshooting wiki, what was the result of running the generated SSH command verbatim outside of VS Code?

I did not try step 2 of the troubleshooting steps

Remote-SSH Log

Description:

The VS Code Server leaves a zombie process () and possibly contributes to lingering ssh-agent processes on the server. The parent node process isn’t reaping its child, and multiple ssh-agent instances persist with parent PID 1, suggesting cleanup issues.

Steps to Reproduce:

Run VS Code Server on Ubuntu via SSH on a cluster. Use ps aux to check processes after activity (e.g., editing, closing). Observe child processes and persistent ssh-agent instances. Actual Behavior:

Full ps output:

user001  2728097       1  0 10:35 ?        00:00:00 /usr/lib/openssh/sftp-server
user002  2695434       1  0 10:04 ?        00:00:00 ssh-agent -s
user003  2360659       1  0 Feb24 ?        00:00:00 ssh-agent -s
user004  2325604       1  0 Feb24 ?        00:00:00 ssh-agent -s
user004  2325030       1  0 Feb24 ?        00:00:00 ssh-agent -s
user004  2299090       1  0 Feb24 ?        00:00:00 ssh-agent -s
user005  2268899       1  0 Feb24 ?        00:00:00 ssh-agent -s
user005  2268273       1  0 Feb24 ?        00:00:00 ssh-agent -s
userxxx  2221109       1  0 Feb24 ?        00:00:00 sh /cluster/home/userxxx/.vscode-server/cli/servers/Stable-138f619c86f1199955d53b4166bef66ef252935c/server/bin/code-server --connection-token=remo
userxxx  2221113 2221109  0 Feb24 ?        00:00:03  \_ /cluster/home/userxxx/.vscode-server/.../node ...
userxxx  2221151 2221113  0 Feb24 ?        00:00:00      \_ /cluster/home/userxxx/.vscode-server/.../node ...
userxxx  2221201 2221113  0 Feb24 ?        00:00:00      \_ [node] <defunct>
user004  2161071       1  0 Feb24 ?        00:00:00 ssh-agent -s

Parent node (PID 2221113) doesn’t reap child (PID 2221201), leaving a zombie. Multiple ssh-agent processes linger.

Expected Behavior:

Parent processes reap children, and SSH sessions clean up, avoiding and orphaned entries.

Environment:

OS: Linux eu-login-01 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 SSH: OpenSSH_8.9p1 Ubuntu-3ubuntu0.11, OpenSSL 3.0.2 15 Mar 2022 VS Code Server: Stable-138f619c86f1199955d53b4166bef66ef252935c Setup: Cluster environment, launched via SSH

this is an old problem never solved see these tickets:

[Remote-SSH Bug]: Remote Terminal Processes Are Not Killed on Exiting VS Code (#9639) Opened: March 5, 2024 Link: https://github.com/microsoft/vscode-remote-release/issues/9639 Relevance: Processes started in the remote terminal persist after closing VS Code, similar to your zombie node. Mentions pkill -9 node disrupting the server, hinting at cleanup failures. Still open, lacking dev response—could overlap with your issue. Three Remote Processes Remain Running After Disconnecting (#6894) Opened: July 4, 2022 Link: https://github.com/microsoft/vscode-remote-release/issues/6894 Relevance: Reports code-server, node, and ptyHost lingering after SSH disconnect, akin to your node and ssh-agent buildup. Unresolved, marked “info-needed”—suggests a pattern of poor process cleanup. Sometimes the 'vscode-server' Process Doesn't Close on Remote End (#1086) Opened: ~2019 (exact date vague), Closed: Not explicitly resolved Link: https://github.com/microsoft/vscode-remote-release/issues/1086 Relevance: Describes a zombie vscode-server process after closing VS Code, blocking reconnection until manually killed—mirrors your node. Closed without a clear fix, suggesting it’s a recurring issue. Visual Studio Code Remote Development Does Not Cleanup All Processes on Remote (#396) Opened: May 20, 2019, Closed: ~2019 Link: https://github.com/microsoft/vscode-remote-release/issues/396 Relevance: Notes multiple extension processes lingering on the remote host, taking memory. Closed, possibly fixed in later releases, but your older server version (Stable-138f619c...) might predate the fix. VsCode Over SSH Can Leave Zombie Processes on Remote Host (#7660) Opened: December 7, 2022, Closed: Unknown resolution Link: https://github.com/microsoft/vscode-remote-release/issues/7660 Relevance: Explicitly calls out zombie VS Code processes after ungraceful SSH termination (e.g., laptop shutdown), matching your cluster scenario. Suggests proactive cleanup on restart—closed, but fix status is unclear.

Expected Behavior

all spawned processes are cleaned up if the ssh session drops.

Actual Behavior

lingering processes

Steps To Reproduce

run ps on server after many vs code shh sessions drops.

Anything else?

No response

zakkg3 avatar Feb 25 '25 17:02 zakkg3

Thank you for compiling a list of prior issues of this unresolved issue!

Problem still exists with vscode 1.100.2 and ms-vscode-remote.vscode-remote-extensionpack 0.26.0.

Even when I manually disconnect from the remote session by clicking 'close remote connection' from the lower-left menu in the vscode gui, they remain and accumulate, at least on remote Ubuntu 22.04 and EL9.

$ ps -ef | grep .vscode-server/cli | wc -l
6

If a system has multiple users and is restarted infrequently, it really adds up, especially with disconnects / reconnects

qhaas avatar May 31 '25 01:05 qhaas