vscode-remote-release
vscode-remote-release copied to clipboard
microsoft
`screen` command extremely slow (~11s) in Remote-SSH terminal due to massive poll(POLLNVAL) loop
Is there an existing issue for this bug?
- [x] I have searched the existing open issues and found none that apply.
- [x] If I find any issue of interest that is related or closed, I will included a link to it in this issue.
Required Troubleshooting Steps
- [x] I have read and performed the troubleshooting steps
- [x] I have tried both values of the
remote.SSH.useLocalServersetting - [x] My issue was not covered in the Tips and Tricks linked from the Troubleshooting Wiki.
- [x] I will include a complete copy of my Remote - SSH logs by running Remote-SSH: Show Log in the command palette or from View > Output in the menu bar
[Optional] Diagnose with Copilot
I asked @remote-ssh but it did not help
In step 2 of the troubleshooting wiki, what was the result of running the generated SSH command verbatim outside of VS Code?
I COULD successfully SSH to the remote machine with the generated SSH command
Remote-SSH Log
Remote-SSH Log
[14:59:58.562] Log Level: 2
[14:59:58.567] SSH Resolver called for "ssh-remote+[remote-ip]", attempt 1
[14:59:58.568] remote.SSH.useLocalServer = true
[14:59:58.568] remote.SSH.useExecServer = true
[14:59:58.568] remote.SSH.bindHost = {}
[14:59:58.568] remote.SSH.path = undefined
[14:59:58.568] remote.SSH.configFile = undefined
[14:59:58.568] remote.SSH.useFlock = true
[14:59:58.568] remote.SSH.lockfilesInTmp = false
[14:59:58.568] remote.SSH.localServerDownload = auto
[14:59:58.569] remote.SSH.remoteServerListenOnSocket = false
[14:59:58.569] remote.SSH.defaultExtensions = []
[14:59:58.569] remote.SSH.defaultExtensionsIfInstalledLocally = []
[14:59:58.569] remote.SSH.loglevel = 2
[14:59:58.569] remote.SSH.enableDynamicForwarding = true
[14:59:58.569] remote.SSH.enableRemoteCommand = false
[14:59:58.569] remote.SSH.serverPickPortsFromRange = {}
[14:59:58.569] remote.SSH.serverInstallPath = {}
[14:59:58.569] remote.SSH.permitPtyAllocation = false
[14:59:58.569] remote.SSH.preferredLocalPortRange = undefined
[14:59:58.569] remote.SSH.useCurlAndWgetConfigurationFiles = false
[14:59:58.569] remote.SSH.experimental.chat = false
[14:59:58.569] remote.SSH.experimental.enhancedSessionLogs = false
[14:59:58.569] remote.SSH.httpProxy = {}
[14:59:58.569] remote.SSH.httpsProxy = {}
[14:59:58.572] VS Code version: 1.99.1
[14:59:58.572] Remote-SSH version: [email protected]
[14:59:58.572] darwin arm64
[14:59:58.578] SSH Resolver called for host: [remote-ip]
[14:59:58.578] Setting up SSH remote "[remote-ip]"
[14:59:58.581] Acquiring local install lock: /var/folders/yy/9hm_5p156j9d8dk50lkvx79w0000gn/T/vscode-remote-ssh-805c0d5d-install.lock
[14:59:58.583] Looking for existing server data file at /Users/myuser/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-805c0d5d-7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b-0.119.0-es/data.json
[14:59:58.583] No existing data file
[14:59:58.583] Using commit id "7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b" and quality "stable" for server
[14:59:58.583] Extensions to install:
[14:59:58.587] Install and start server if needed
[14:59:58.588] PATH: /Users/myuser/.composer/vendor/bin:/opt/homebrew/opt/ruby/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/myuser/dev/flutter/bin:/Users/myuser/Library/Android/sdk/platform-tools:/Users/myuser/dev/flutter/.pub-cache/bin:/Users/myuser/.pub-cache/bin:/Users/myuser/dev/Google Protobuf/protoc-3.19.3-osx-aarch_64
[14:59:58.589] Checking ssh with "ssh -V"
[14:59:58.595] > OpenSSH_9.9p1, LibreSSL 3.3.6
[14:59:58.598] askpass server listening on /var/folders/yy/9hm_5p156j9d8dk50lkvx79w0000gn/T/vscode-ssh-askpass-13ba33f2b9749580d72c491606f74601a50b303f.sock
[14:59:58.598] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/yy/9hm_5p156j9d8dk50lkvx79w0000gn/T/vscode-ssh-askpass-9a766822286f51084fc93b2c69af869af378cb1c.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","55971","-o","ConnectTimeout=15","[remote-ip]"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/myuser/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-805c0d5d-7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b-0.119.0-es/data.json"}
[14:59:58.598] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XqtfNAnnTj/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.ASk3e43Edt/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/myuser/.vscode/extensions/ms-vscode-remote.remote-ssh-0.119.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/myuser/.vscode/extensions/ms-vscode-remote.remote-ssh-0.119.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/yy/9hm_5p156j9d8dk50lkvx79w0000gn/T/vscode-ssh-askpass-13ba33f2b9749580d72c491606f74601a50b303f.sock"}
[14:59:58.598] Spawned 29657
[14:59:58.599] Using connect timeout of 17 seconds
[14:59:58.675] > local-server-1> Running ssh connection command: ssh -v -T -D 55971 -o ConnectTimeout=15 [remote-ip]
[14:59:58.676] > local-server-1> Spawned ssh, pid=29664
[14:59:58.680] stderr> OpenSSH_9.9p1, LibreSSL 3.3.6
[14:59:58.800] stderr> debug1: Server host key: ssh-ed25519 SHA256:3Ff4LaVKREWWHDvU7W4eTTticpkRVTH/niWCLiAJqP0
[14:59:58.920] stderr> Authenticated to [remote-ip] ([[remote-ip]]:22) using "publickey".
[14:59:59.446] > Welcome to Ubuntu 24.10 (GNU/Linux 6.11.0-21-generic x86_64)
>
> * Documentation: https://help.ubuntu.com
> * Management: https://landscape.canonical.com
> * Support: https://ubuntu.com/pro
>
> System information as of Thu Apr 10 02:59:59 PM CEST 2025
>
> System load: 0.24
> Usage of /: 16.6% of 228.00GB
> Memory usage: 10%
> Swap usage: 0%
> Temperature: 54.0 C
> Processes: 212
> Users logged in: 1
> IPv4 address for enxc4411e7426b8: [remote-ip]
> IPv6 address for enxc4411e7426b8: fda8:7a38:a8b9:345:c641:1eff:fe74:26b8
>
> * Strictly confined Kubernetes makes edge and IoT secure. Learn how MicroK8s
> just raised the bar for easy, resilient and secure K8s cluster deployment.
>
> https://ubuntu.com/engage/secure-kubernetes-at-the-edge
>
> 0 updates can be applied immediately.
>
[14:59:59.456] > ready: 388b2accd28d
[14:59:59.463] > Linux 6.11.0-21-generic #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 19 16:50:40 UTC 2025
[14:59:59.464] Platform: linux
[14:59:59.563] > /bin/bash
[14:59:59.564] Parent Shell: bash
[14:59:59.564] Parent Shell pid: 29657
[14:59:59.565] Waiting for subshell to start
[14:59:59.580] > 118081
[14:59:59.580] stdout -> '118081'
[14:59:59.580] sub-process detected
[14:59:59.601] > 388b2accd28d: running
> Script executing under PID: 118081
[14:59:59.605] > Found existing installation at /home/myuser/.vscode-server...
> Starting VS Code CLI...
> Removing old logfile at /home/myuser/.vscode-server/.cli.7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b.log
[14:59:59.608] > Spawned remote CLI: 118099
[14:59:59.612] > Waiting for server log...
[14:59:59.646] > Waiting for server log...
[14:59:59.684] > 388b2accd28d: start
> listeningOn==127.0.0.1:38391==
> osReleaseId==ubuntu==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/run/user/1000==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> serverStartTime==77==
> execServerToken==1a111111-1111-111a-a1a1-11aaa11a1111==
> platformDownloadPath==cli-alpine-x64==
> SSH_AUTH_SOCK==/tmp/ssh-iqQqUYXNBi/agent.118074==
> DISPLAY====
> 388b2accd28d: end
[14:59:59.684] Received install output:
listeningOn==127.0.0.1:38391==
osReleaseId==ubuntu==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/run/user/1000==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
serverStartTime==77==
execServerToken==1a111111-1111-111a-a1a1-11aaa11a1111==
platformDownloadPath==cli-alpine-x64==
SSH_AUTH_SOCK==/tmp/ssh-iqQqUYXNBi/agent.118074==
DISPLAY====
[14:59:59.684] Remote server is listening on port 38391
[14:59:59.684] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":38391},"osReleaseId":"ubuntu","arch":"x86_64","sshAuthSock":"/tmp/ssh-iqQqUYXNBi/agent.118074","display":"","tmpDir":"/run/user/1000","platform":"linux","execServerToken":"1a111111-1111-111a-a1a1-11aaa11a1111"},"serverStartTime":77,"installUnpackCode":""}
[14:59:59.685] Persisting server connection details to /Users/myuser/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-805c0d5d-7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b-0.119.0-es/data.json
[14:59:59.687] Starting forwarding server. local port 55973 -> socksPort 55971 -> remotePort 38391
[14:59:59.687] Forwarding server listening on port 55973
[14:59:59.687] Waiting for ssh tunnel to be ready
[14:59:59.688] Tunneled port 38391 to local port 55973
[14:59:59.688] Resolved "ssh-remote+[remote-ip]" to "port 55973"
[14:59:59.692] Initizing new exec server for ssh-remote+[remote-ip]
[14:59:59.692] Resolving exec server at port 55973
[14:59:59.693] [Forwarding server port 55973] Got connection 0
[14:59:59.693] [Forwarding server port 55973] Got connection 1
[14:59:59.765] Exec server for ssh-remote+[remote-ip] created and cached
[14:59:59.765] Extensions to install:
[14:59:59.765] Updating $SSH_AUTH_SOCK: ln -f -s "/tmp/ssh-iqQqUYXNBi/agent.118074" "/run/user/1000/vscode-ssh-auth-sock-828200207"
[14:59:59.766] Using cwd: vscode-remote://ssh-remote%2B[remote-ip]/
[14:59:59.766] Remote extension host environment: {"SSH_AUTH_SOCK":"/run/user/1000/vscode-ssh-auth-sock-828200207"}
[14:59:59.768] ------
[14:59:59.769] No hints found in the recent session.
[14:59:59.792] [server] Checking /home/myuser/.vscode-server/cli/servers/Stable-7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b/log.txt and /home/myuser/.vscode-server/cli/servers/Stable-7c6fdfb0b8f2f675eb0b47f3d95eeca78962565b/pid.txt for a running server...
[14:59:59.830] [server] Found running server (pid=117266)
[15:00:01.072] Opening exec server for ssh-remote+[remote-ip]
[15:00:01.078] Opening exec server for ssh-remote+[remote-ip]
[15:00:01.130] Verified and reusing cached exec server for ssh-remote+[remote-ip]
[15:00:01.130] Verified and reusing cached exec server for ssh-remote+[remote-ip]
[15:00:01.130] No hints found in the recent session.
[15:00:01.131] No hints found in the recent session.
[15:00:01.140] Opening exec server for ssh-remote+[remote-ip]
[15:00:01.169] Verified and reusing cached exec server for ssh-remote+[remote-ip]
[15:00:01.169] No hints found in the recent session.
[15:00:01.238] "Update SSH_AUTH_SOCK" terminal command done
Expected Behavior
When running the command screen -v within the integrated terminal of a VS Code Remote-SSH session, it is expected to execute almost instantly (e.g., within milliseconds) and display the GNU Screen version information.
This is the behavior observed when running the same command on the same remote host via a standard external terminal (like macOS Terminal). There should be no significant startup delay.
Actual Behavior
When running the command screen -v within the integrated terminal of a VS Code Remote-SSH session, there is a consistent delay of approximately 10-11 seconds before the version information is displayed and the command exits.
Diagnostics using strace -c revealed that virtually 100% of this delay (~10.7 seconds) is spent executing the poll() system call over one million times. Deeper tracing showed these poll() calls are made with large arrays of apparently invalid file descriptors, causing the kernel to return POLLNVAL immediately for each descriptor in the set. The process appears stuck in this poll(POLLNVAL) loop for the duration of the delay.
This slow behaviour occurs only within the VS Code Remote-SSH integrated terminal environment and persists regardless of whether the remote.SSH.useLocalServer setting is true or false.
Steps To Reproduce
- Connect to a remote host running Ubuntu Server (tested on Ubuntu 24.10, kernel 6.11.0-21-generic) using VS Code Remote-SSH (Extension version 0.119.0, VS Code 1.99.1).
- Ensure the
screenpackage is installed on the remote host (sudo apt update && sudo apt install screen). Version tested:4.09.01. - Ensure the standard SSH server configuration using PAM is active (
UsePAM yesin/etc/ssh/sshd_config). - Open the integrated terminal within the VS Code Remote-SSH session (Terminal > New Terminal).
- Run the command:
time screen -v - Observe the
realexecution time reported by thetimecommand (it should be ~10-11 seconds).
Anything else?
The primary context is the poll(POLLNVAL) loop detailed under 'Actual Behaviour', which seems isolated to the VS Code Server environment. Full logs provided; the large (~2GB) strace log file is available upon request. Happy to run further diagnostics if needed.
