vscode-remote-release
vscode-remote-release copied to clipboard
[Remote-SSH Bug]: `SSH_AUTH_SOCK` is getting stale over time
Is there an existing issue for this bug?
- [X] I have searched the existing issues
Required Troubleshooting Steps
- [X] I have followed these troubleshooting steps
- [X] I have tried both values of the
remote.SSH.useLocalServer
setting
Connect Locally
It connects successfully
->
Over time (I presume when the system goes to sleep and ssh connection to the remote gets disconnected), the ssh agent socket file - $SSH_AUTH_SOCK
- starts to point to a non-existing file.
Example:
❯ file $SSH_AUTH_SOCK
/tmp/ssh-XXXX5pnG8t/agent.662781: cannot open `/tmp/ssh-XXXX5pnG8t/agent.662781' (No such file or directory)
This, in its turn, makes the SSH agent unavailable in the remote system and the only workaround is to "kill the remote server" which is very annoying.
Expected Behavior
SSH_AUTH_SOCK
refreshes when the remote is reconnected.
Steps To Reproduce
- use SSH agent forwarding
- make system go to sleep
- after reconnecting to the remote check if the
file $SSH_AUTH_SOCK
resolves to an existing file
Remote-SSH Log
Remote-SSH Log
[09:23:17.216] Log Level: 2
[09:23:17.222] VS Code version: 1.88.0
[09:23:17.222] Remote-SSH version: [email protected]
[09:23:17.223] win32 x64
[09:23:17.228] SSH Resolver called for "ssh-remote+example", attempt 1
[09:23:17.229] "remote.SSH.useLocalServer": false
[09:23:17.229] "remote.SSH.useExecServer": true
[09:23:17.229] "remote.SSH.showLoginTerminal": false
[09:23:17.229] "remote.SSH.remotePlatform": {"example":"linux"}
[09:23:17.229] "remote.SSH.path": C:\Windows\System32\OpenSSH\ssh.exe
[09:23:17.229] "remote.SSH.configFile": undefined
[09:23:17.230] "remote.SSH.useFlock": true
[09:23:17.231] "remote.SSH.lockfilesInTmp": false
[09:23:17.231] "remote.SSH.localServerDownload": auto
[09:23:17.231] "remote.SSH.remoteServerListenOnSocket": false
[09:23:17.231] "remote.SSH.showLoginTerminal": false
[09:23:17.231] "remote.SSH.defaultExtensions": []
[09:23:17.231] "remote.SSH.loglevel": 2
[09:23:17.232] "remote.SSH.enableDynamicForwarding": true
[09:23:17.232] "remote.SSH.enableRemoteCommand": true
[09:23:17.232] "remote.SSH.serverPickPortsFromRange": {}
[09:23:17.232] "remote.SSH.serverInstallPath": {}
[09:23:17.238] SSH Resolver called for host: example
[09:23:17.238] Setting up SSH remote "example"
[09:23:17.242] Using commit id "5c3e652f63e798a5ac2f31ffd0d863669328dc4c" and quality "stable" for server
[09:23:17.245] Install and start server if needed
[09:23:17.254] Checking ssh with "C:\Windows\System32\OpenSSH\ssh.exe -V"
[09:23:17.288] > OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
[09:23:17.292] Running script with connection command: "C:\Windows\System32\OpenSSH\ssh.exe" -T -D 63708 example bash
[09:23:17.294] Terminal shell path: C:\Windows\System32\cmd.exe
[09:23:18.521] > �]0;C:\Windows\System32\cmd.exe�
[09:23:18.521] Got some output, clearing connection timeout
[09:23:20.686] > ac26998f326f: running```
Anything else?
Related issue - https://github.com/microsoft/vscode-remote-release/issues/9767 - was closed because of the workaround, but doesn't actually provided a long term fix.
This issue is meant to track the cause of the issue.
/cc @danielhass
@hellt thanks for re-opening this as a new issue!
We are seeing same behavior. From Windows or MacOS VS Code client (latest version). We connect to a Linux RHEL8 host with ssh agent forwarding. First remote-ssh connection succeeds. Then, if you "Developer: Reload Window" or if you "Close Remote Connection" and reconnect again, the env var $SSH_SOCK_AUTH refers to an invalid socket file. You can confirm this by starting a new terminal and typing file $SSH_SOCK_AUTH
. You will then get an error that looks like this /tmp/ssh-Axx5CAtS6J/agent.1749639: cannot open '/tmp/ssh-Axx5CAtS6J/agent.1749639' (No such file or directory)
Furthermore, if you use SSH to acces git repos within the SCM, you will get "access denied"
The "solution" is that you have to kill remote vscode server and reconnect again to have succesfull forwarding.
Workarounds I found was :
- Downgrade Remote-SSH extension to v0.102.0 (+1 year old)
- Set "remote.SSH.useExecServer": false in settings.json
VS Code Version: 1.89.1 Remote-SSH Version: v0.110.1
The workaround specified to set "remote.SSH.useExecServer": false
makes things work again for me.
I faced this issue on vscode on the remote machine, where was I fixing the tmux.conf (pertaining to SSH key forwarding) & noticed this SSH_AUTH_SOCK stale issue. setting "remote.SSH.useExecServer": false resolved the issue.
Thanks for the issue, fixed in the next SSH prerelease version
Hi @hellt! Are you able to verify if this fix works? You can do so by getting the latest pre-release version of the remote-ssh extension and trying these same steps there. We want to make sure we get this bug verified, and the fix is correct, before we release this to stable. Thanks!
Hi @eleanorjboyd
yeah, I can do that
I believe I should also revert the change done to remote.SSH.useLocalServer
Hi @eleanorjboyd, thanks for asking back on this one.
Cloud you let us know the specific pre-release version number that includes the possible fix you are referencing? This would allow us to make sure we're testing against the correct codebase. Thx!
The latest version (v0.113.2024072315) should contain the fixes
You might also need to use VS Code Insiders to access the newest pre-release version of Remote-SSH.
@connor4312 @eleanorjboyd
it seems it doesn't solve it for me. Running v0.113.2024072315
but once the ssh session idled enough to be disconnected, upon reconnection the socket path is stale.
Here is what I have after I reconnected using vscode:
❯ echo $SSH_AUTH_SOCK
/run/user/1000/vscode-ssh-auth-sock-424878423
❯ file $SSH_AUTH_SOCK
/run/user/1000/vscode-ssh-auth-sock-424878423: broken symbolic link to /tmp/ssh-XXXXFcu178/agent.25563
the symlink is broken because the target socket doesn't exist
❯ file /tmp/ssh-XXXXFcu178/agent.25563
/tmp/ssh-XXXXFcu178/agent.25563: cannot open `/tmp/ssh-XXXXFcu178/agent.25563' (No such file or directory)
What exists is:
❯ ls /tmp/ssh*
/tmp/ssh-XXXX3cxDPM:
agent.25412
/tmp/ssh-XXXXP3dAKB:
agent.19566
If you're keeping a terminal open to check, please watch for any environment variable updates in that terminal (signalled by the ⚠️ ) or open a new terminal which will receive any refreshed variables.
not sure what you mean, tbh. opening a new terminal doesn't solve it for me, it uses the same symlink to a non existing socket file
My current workaround is to run this oneliner whenever the socket var gets stale:
export SSH_AUTH_SOCK=$(find /tmp/ssh* -type s -exec test -S {} \; -print)
Can you share the output of your Remote - SSH log?
@connor4312 if that is the one in the OUTPUT tab of the vscode:
[20:09:08.617] Log Level: 2
[20:09:08.626] VS Code version: 1.91.1
[20:09:08.626] Remote-SSH version: [email protected]
[20:09:08.626] darwin arm64
[20:09:08.627] SSH Resolver called for "ssh-remote+devbox", attempt 1
[20:09:08.627] "remote.SSH.useLocalServer": true
[20:09:08.627] "remote.SSH.useExecServer": true
[20:09:08.627] "remote.SSH.path": undefined
[20:09:08.628] "remote.SSH.configFile": undefined
[20:09:08.628] "remote.SSH.useFlock": true
[20:09:08.628] "remote.SSH.lockfilesInTmp": false
[20:09:08.628] "remote.SSH.localServerDownload": auto
[20:09:08.628] "remote.SSH.remoteServerListenOnSocket": false
[20:09:08.628] "remote.SSH.showLoginTerminal": false
[20:09:08.628] "remote.SSH.defaultExtensions": []
[20:09:08.628] "remote.SSH.loglevel": 2
[20:09:08.628] "remote.SSH.enableDynamicForwarding": true
[20:09:08.628] "remote.SSH.enableRemoteCommand": false
[20:09:08.628] "remote.SSH.serverPickPortsFromRange": {}
[20:09:08.628] "remote.SSH.serverInstallPath": {}
[20:09:08.628] "remote.SSH.permitPtyAllocation": false
[20:09:08.628] "remote.SSH.preferredLocalPortRange: undefined
[20:09:08.628] "remote.SSH.useCurlAndWgetConfigurationFiles: false
[20:09:08.636] SSH Resolver called for host: devbox
[20:09:08.636] Setting up SSH remote "devbox"
[20:09:08.638] Acquiring local install lock: /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-remote-ssh-c4ea1055-install.lock
[20:09:08.638] Looking for existing server data file at /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json
[20:09:08.639] Using commit id "f1e16e1e6214d7c44d078b1f0607b2388f29d729" and quality "stable" for server
[20:09:08.641] Install and start server if needed
[20:09:08.644] PATH: /Users/romandodin/.rd/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/go/bin:/Users/romandodin/.orbstack/bin:/usr/local/bin
[20:09:08.644] Checking ssh with "ssh -V"
[20:09:08.650] > OpenSSH_9.6p1, LibreSSL 3.3.6
[20:09:08.653] askpass server listening on /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-d8f4cb87fd2de011be3bff1d8b549633d2da40a5.sock
[20:09:08.657] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-430df199eb864e2f4933b3c8397ce7a427bbf9fe.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","51620","-o","ConnectTimeout=60","devbox","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json"}
[20:09:08.657] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.N2ibEDNoeQ/Listeners","SHELL":"/bin/zsh","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-d8f4cb87fd2de011be3bff1d8b549633d2da40a5.sock"}
[20:09:08.657] Spawned 25191
[20:09:08.735] > local-server-1> Running ssh connection command: ssh -v -T -D 51620 -o ConnectTimeout=60 devbox bash
[20:09:08.737] > local-server-1> Spawned ssh, pid=25197
[20:09:08.740] stderr> OpenSSH_9.6p1, LibreSSL 3.3.6
[20:09:08.965] stderr> debug1: Server host key: ssh-ed25519 SHA256:bqazz6I8kUfoqbL9tvwFI1AV01uW25hzTa7yVSG0BZA
[20:09:09.587] stderr> Authenticated to devbox.panda-cobra.ts.net ([100.72.255.47]:22) using "publickey".
[20:09:09.800] > ready: 12a1359113f8
[20:09:09.847] > Linux 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03)
[20:09:09.848] Platform: linux
[20:09:09.889] > /usr/bin/zsh
[20:09:09.889] Shell: zsh
[20:09:09.889] Creating bash subshell inside "zsh"
[20:09:09.933] > bash version: 5.2.15(1)-release
[20:09:09.991] > 12a1359113f8: running
[20:09:10.011] > Found existing installation at /home/rd/.vscode-server...
[20:09:10.014] > Starting VS Code CLI... "/home/rd/.vscode-server/code-f1e16e1e6214d7c44d078b1f0607b2388f29d729" command-shell --cli-data-dir "/home/rd/.vscode-server/cli" --on-port --on-host=127.0.0.1 --parent-process-id 32747 &> "/home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log" < /dev/null
[20:09:10.017] > printenv:
> SHELL=/usr/bin/zsh
> SSH_AUTH_SOCK=/tmp/ssh-XXXXcx8RxM/agent.32744
> PWD=/home/rd
> LOGNAME=rd
> XDG_SESSION_TYPE=tty
> MOTD_SHOWN=pam
> HOME=/home/rd
> LANG=en_US.UTF-8
> VSCODE_AGENT_FOLDER=/home/rd/.vscode-server
> SSH_CONNECTION=100.127.252.185 51622 100.72.255.47 22
> XDG_SESSION_CLASS=user
> USER=rd
> SHLVL=2
> XDG_SESSION_ID=550
> XDG_RUNTIME_DIR=/run/user/1000
> ARTIFACTORY_TOKEN=cmVmdGtuOjAxOjE3NDE4MDIxMjM6ZjJwYjRTSXAzQXU3Q01nQzg2bW1jSnJOc3N2
> SSH_CLIENT=100.127.252.185 51622 22
> PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/go/bin:/home/rd/go/bin:/home/rd/.pulumi/bin
> DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> OLDPWD=/home/rd
> _=/usr/bin/printenv
[20:09:10.017] > Removing old logfile at /home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log
[20:09:10.018] > Spawned remote CLI: 32766
[20:09:10.049] > Waiting for server log...
[20:09:10.088] > 12a1359113f8: start
> listeningOn==127.0.0.1:35607==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/run/user/1000==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> serverStartTime==75==
> execServerToken==11aa1a11-aa1a-1a11-111a-1aaaa1a11111==
[20:09:10.088] > 12a1359113f8: end
[20:09:10.088] Received install output:
listeningOn==127.0.0.1:35607==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/run/user/1000==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
serverStartTime==75==
execServerToken==11aa1a11-aa1a-1a11-111a-1aaaa1a11111==
[20:09:10.088] Remote server is listening on port 35607
[20:09:10.088] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":35607},"osReleaseId":"debian","arch":"x86_64","tmpDir":"/run/user/1000","platform":"linux","execServerToken":"11aa1a11-aa1a-1a11-111a-1aaaa1a11111"},"serverStartTime":75,"installUnpackCode":""}
[20:09:10.089] Persisting server connection details to /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json
[20:09:10.094] Starting forwarding server. local port 51623 -> socksPort 51620 -> remotePort 35607
[20:09:10.094] Forwarding server listening on port 51623
[20:09:10.094] Waiting for ssh tunnel to be ready
[20:09:10.096] Tunneled port 35607 to local port 51623
[20:09:10.096] Resolved "ssh-remote+devbox" to "port 51623"
[20:09:10.103] Initizing new exec server for ssh-remote+devbox
[20:09:10.103] Resolving exec server at port 51623
[20:09:10.119] [Forwarding server port 51623] Got connection 0
[20:09:10.122] [Forwarding server port 51623] Got connection 1
[20:09:10.261] Exec server for ssh-remote+devbox created and cached
[20:09:10.268] ------
[20:09:10.319] [server] Checking /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/log.txt and /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/pid.txt for a running server...
[20:09:10.319] [server] Found running server (pid=9256)
[20:09:15.938] Opening exec server for ssh-remote+devbox
[20:09:15.973] Opening exec server for ssh-remote+devbox
[20:09:17.197] Verified and reusing cached exec server for ssh-remote+devbox
[20:09:17.201] Verified and reusing cached exec server for ssh-remote+devbox
[20:09:17.247] Opening exec server for ssh-remote+devbox
[20:09:17.304] Verified and reusing cached exec server for ssh-remote+devbox
In those logs it doesn't look like SSH forwarding is on in your settings (probably remote.SSH.enableAgentForwarding
), so I would not expect it to work there.
If agent forwarding is enabled you should see the substring SSH_AUTH_SOCK==
in your output (empty if ForwardAgent != yes in your config)
@connor4312 @eleanorjboyd it seems it doesn't solve it for me. Running
v0.113.2024072315
but once the ssh session idled enough to be disconnected, upon reconnection the socket path is stale.Here is what I have after I reconnected using vscode:
❯ echo $SSH_AUTH_SOCK /run/user/1000/vscode-ssh-auth-sock-424878423 ❯ file $SSH_AUTH_SOCK /run/user/1000/vscode-ssh-auth-sock-424878423: broken symbolic link to /tmp/ssh-XXXXFcu178/agent.25563
the symlink is broken because the target socket doesn't exist
❯ file /tmp/ssh-XXXXFcu178/agent.25563 /tmp/ssh-XXXXFcu178/agent.25563: cannot open `/tmp/ssh-XXXXFcu178/agent.25563' (No such file or directory)
What exists is:
❯ ls /tmp/ssh* /tmp/ssh-XXXX3cxDPM: agent.25412 /tmp/ssh-XXXXP3dAKB: agent.19566
You'd have to manually reestablish the socket upon reconnect, f.e. in ~/.ssh/rc
script.
In those logs it doesn't look like SSH forwarding is on in your settings (probably
remote.SSH.enableAgentForwarding
), so I would not expect it to work there.If agent forwarding is enabled you should see the substring
SSH_AUTH_SOCK==
in your output (empty if ForwardAgent != yes in your config)
In my vscode config I tried both with Enable Agent Forwarding
and without it.
In my ~/.ssh/config
I have agentforwarding set for all hosts, that is why I have a socket under /tmp/ssh*
. The problem is that vscode started to not have the $SSH_AUTH_SOCK update when the session is reconnected.
Happy to provide more logs if there is a guide how to collect them.
ADD1:
Here is the log with Enable Agent Forwarding
enabled
[23:24:14.728] Log Level: 2 [23:24:14.737] VS Code version: 1.91.1 [23:24:14.737] Remote-SSH version: [email protected] [23:24:14.737] darwin arm64 [23:24:14.738] SSH Resolver called for "ssh-remote+devbox", attempt 1 [23:24:14.738] "remote.SSH.useLocalServer": true [23:24:14.738] "remote.SSH.useExecServer": true [23:24:14.739] "remote.SSH.path": undefined [23:24:14.739] "remote.SSH.configFile": undefined [23:24:14.739] "remote.SSH.useFlock": true [23:24:14.739] "remote.SSH.lockfilesInTmp": false [23:24:14.739] "remote.SSH.localServerDownload": auto [23:24:14.739] "remote.SSH.remoteServerListenOnSocket": false [23:24:14.739] "remote.SSH.showLoginTerminal": false [23:24:14.739] "remote.SSH.defaultExtensions": [] [23:24:14.739] "remote.SSH.loglevel": 2 [23:24:14.739] "remote.SSH.enableDynamicForwarding": true [23:24:14.739] "remote.SSH.enableRemoteCommand": false [23:24:14.739] "remote.SSH.serverPickPortsFromRange": {} [23:24:14.739] "remote.SSH.serverInstallPath": {} [23:24:14.739] "remote.SSH.permitPtyAllocation": false [23:24:14.739] "remote.SSH.preferredLocalPortRange: undefined [23:24:14.739] "remote.SSH.useCurlAndWgetConfigurationFiles: false [23:24:14.748] SSH Resolver called for host: devbox [23:24:14.748] Setting up SSH remote "devbox" [23:24:14.750] Acquiring local install lock: /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-remote-ssh-c4ea1055-install.lock [23:24:14.751] Looking for existing server data file at /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json [23:24:14.751] Using commit id "f1e16e1e6214d7c44d078b1f0607b2388f29d729" and quality "stable" for server [23:24:14.753] Install and start server if needed [23:24:14.757] PATH: /Users/romandodin/.rd/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/go/bin:/Users/romandodin/.orbstack/bin:/usr/local/bin [23:24:14.757] Checking ssh with "ssh -V" [23:24:14.763] > OpenSSH_9.6p1, LibreSSL 3.3.6[23:24:14.766] askpass server listening on /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-0a61126baf8f16e0cbf61b8fb399756e7a398827.sock [23:24:14.767] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-731623ca2182b5077549174466bacf85edc95bb6.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","59026","-o","ConnectTimeout=60","devbox","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json"} [23:24:14.768] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.N2ibEDNoeQ/Listeners","SHELL":"/bin/zsh","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-0a61126baf8f16e0cbf61b8fb399756e7a398827.sock"} [23:24:14.768] Spawned 48365 [23:24:14.849] > local-server-1> Running ssh connection command: ssh -v -T -D 59026 -o ConnectTimeout=60 devbox bash [23:24:14.850] > local-server-1> Spawned ssh, pid=48371 [23:24:14.854] stderr> OpenSSH_9.6p1, LibreSSL 3.3.6 [23:24:15.086] stderr> debug1: Server host key: ssh-ed25519 SHA256:bqazz6I8kUfoqbL9tvwFI1AV01uW25hzTa7yVSG0BZA [23:24:15.742] stderr> Authenticated to devbox.panda-cobra.ts.net ([100.72.255.47]:22) using "publickey". [23:24:15.918] > ready: b85fe4112dbc [23:24:15.961] > Linux 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) [23:24:15.961] Platform: linux [23:24:16.002] > /usr/bin/zsh [23:24:16.002] Shell: zsh [23:24:16.002] Creating bash subshell inside "zsh" [23:24:16.044] > bash version: 5.2.15(1)-release [23:24:16.105] > b85fe4112dbc: running [23:24:16.119] > Found existing installation at /home/rd/.vscode-server...
Starting VS Code CLI... "/home/rd/.vscode-server/code-f1e16e1e6214d7c44d078b1f0607b2388f29d729" command-shell --cli-data-dir "/home/rd/.vscode-server/cli" --on-port --on-host=127.0.0.1 --parent-process-id 15888 &> "/home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log" < /dev/null printenv: [23:24:16.122] > SHELL=/usr/bin/zsh SSH_AUTH_SOCK=/tmp/ssh-XXXXgE67vl/agent.15885 PWD=/home/rd LOGNAME=rd XDG_SESSION_TYPE=tty MOTD_SHOWN=pam HOME=/home/rd LANG=en_US.UTF-8 VSCODE_AGENT_FOLDER=/home/rd/.vscode-server SSH_CONNECTION=100.127.252.185 59027 100.72.255.47 22 XDG_SESSION_CLASS=user USER=rd SHLVL=2 XDG_SESSION_ID=555 XDG_RUNTIME_DIR=/run/user/1000 ARTIFACTORY_TOKEN=cmVmdGtuOjAxOjE3NDE4MDIxMjM6ZjJwYjRTSXAzQXU3Q01nQzg2bW1jSnJOc3N2 SSH_CLIENT=100.127.252.185 59027 22 PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/go/bin:/home/rd/go/bin:/home/rd/.pulumi/bin DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus OLDPWD=/home/rd _=/usr/bin/printenv [23:24:16.127] > Removing old logfile at /home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log Spawned remote CLI: 15907 [23:24:16.161] > Waiting for server log... [23:24:16.196] > b85fe4112dbc: start SSH_AUTH_SOCK==/tmp/ssh-XXXXgE67vl/agent.15885== listeningOn==127.0.0.1:42393== osReleaseId==debian== [23:24:16.197] > arch==x86_64== vscodeArch==x64== bitness==64== tmpDir==/run/user/1000== platform==linux== unpackResult==== didLocalDownload==0== downloadTime==== installTime==== serverStartTime==73== execServerToken==aaa11111-1a1a-111a-111a-aa111a111a1a== b85fe4112dbc: end [23:24:16.197] Received install output: SSH_AUTH_SOCK==/tmp/ssh-XXXXgE67vl/agent.15885== listeningOn==127.0.0.1:42393== osReleaseId==debian==arch==x86_64== vscodeArch==x64== bitness==64== tmpDir==/run/user/1000== platform==linux== unpackResult==== didLocalDownload==0== downloadTime==== installTime==== serverStartTime==73== execServerToken==aaa11111-1a1a-111a-111a-aa111a111a1a==
[23:24:16.197] Remote server is listening on port 42393 [23:24:16.197] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":42393},"osReleaseId":"debian","arch":"x86_64","sshAuthSock":"/tmp/ssh-XXXXgE67vl/agent.15885","tmpDir":"/run/user/1000","platform":"linux","execServerToken":"aaa11111-1a1a-111a-111a-aa111a111a1a"},"serverStartTime":73,"installUnpackCode":""} [23:24:16.197] Persisting server connection details to /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json [23:24:16.205] Starting forwarding server. local port 59028 -> socksPort 59026 -> remotePort 42393 [23:24:16.205] Forwarding server listening on port 59028 [23:24:16.205] Waiting for ssh tunnel to be ready [23:24:16.207] Tunneled port 42393 to local port 59028 [23:24:16.207] Resolved "ssh-remote+devbox" to "port 59028" [23:24:16.213] Initizing new exec server for ssh-remote+devbox [23:24:16.214] Resolving exec server at port 59028 [23:24:16.227] [Forwarding server port 59028] Got connection 0 [23:24:16.230] [Forwarding server port 59028] Got connection 1 [23:24:16.361] Exec server for ssh-remote+devbox created and cached [23:24:16.362] Updating $SSH_AUTH_SOCK: ln -f -s "/tmp/ssh-XXXXgE67vl/agent.15885" "/run/user/1000/vscode-ssh-auth-sock-719961288" [23:24:16.362] Using cwd: vscode-remote://ssh-remote%2Bdevbox/ [23:24:16.363] Remote extension host environment: {"SSH_AUTH_SOCK":"/run/user/1000/vscode-ssh-auth-sock-719961288"} [23:24:16.368] ------
[23:24:16.417] [server] Checking /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/log.txt and /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/pid.txt for a running server... [23:24:16.417] [server] Found running server (pid=9256) [23:24:18.984] "Update SSH_AUTH_SOCK" terminal command done [23:24:18.995] Opening exec server for ssh-remote+devbox [23:24:19.010] Opening exec server for ssh-remote+devbox [23:24:19.152] Verified and reusing cached exec server for ssh-remote+devbox [23:24:19.155] Verified and reusing cached exec server for ssh-remote+devbox [23:24:19.214] Opening exec server for ssh-remote+devbox [23:24:19.258] Verified and reusing cached exec server for ssh-remote+devbox
I will let this session to go stale and check the log upon reconnect
ADD2:
I reconnected after the session got stale and this time around it seems all correct. I see the SSH_AUTH_SOCK updated (as seen from the logs below)
[11:10:56.151] Log Level: 2 [11:10:56.159] VS Code version: 1.91.1 [11:10:56.159] Remote-SSH version: [email protected] [11:10:56.159] darwin arm64 [11:10:56.160] SSH Resolver called for "ssh-remote+devbox", attempt 1 [11:10:56.161] "remote.SSH.useLocalServer": true [11:10:56.161] "remote.SSH.useExecServer": true [11:10:56.161] "remote.SSH.path": undefined [11:10:56.161] "remote.SSH.configFile": undefined [11:10:56.161] "remote.SSH.useFlock": true [11:10:56.161] "remote.SSH.lockfilesInTmp": false [11:10:56.161] "remote.SSH.localServerDownload": auto [11:10:56.161] "remote.SSH.remoteServerListenOnSocket": false [11:10:56.161] "remote.SSH.showLoginTerminal": false [11:10:56.161] "remote.SSH.defaultExtensions": [] [11:10:56.161] "remote.SSH.loglevel": 2 [11:10:56.161] "remote.SSH.enableDynamicForwarding": true [11:10:56.161] "remote.SSH.enableRemoteCommand": false [11:10:56.161] "remote.SSH.serverPickPortsFromRange": {} [11:10:56.161] "remote.SSH.serverInstallPath": {} [11:10:56.161] "remote.SSH.permitPtyAllocation": false [11:10:56.161] "remote.SSH.preferredLocalPortRange: undefined [11:10:56.161] "remote.SSH.useCurlAndWgetConfigurationFiles: false [11:10:56.180] SSH Resolver called for host: devbox [11:10:56.180] Setting up SSH remote "devbox" [11:10:56.182] Acquiring local install lock: /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-remote-ssh-c4ea1055-install.lock [11:10:56.182] Looking for existing server data file at /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json [11:10:56.182] Using commit id "f1e16e1e6214d7c44d078b1f0607b2388f29d729" and quality "stable" for server [11:10:56.184] Install and start server if needed [11:10:56.188] PATH: /Users/romandodin/.rd/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/go/bin:/Users/romandodin/.orbstack/bin:/usr/local/bin [11:10:56.188] Checking ssh with "ssh -V" [11:10:56.209] > OpenSSH_9.6p1, LibreSSL 3.3.6[11:10:56.212] askpass server listening on /var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-b814f6ec04f38bbaec2d4ceacbd3be8d1447260c.sock [11:10:56.213] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-0ab51f3e21cc608b525053a39e755c0edfe63d03.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","49533","-o","ConnectTimeout=60","devbox","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json"} [11:10:56.213] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.N2ibEDNoeQ/Listeners","SHELL":"/bin/zsh","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/romandodin/.vscode/extensions/ms-vscode-remote.remote-ssh-0.113.2024072315/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/lp/1qf36nz164n99mlvys3rv04m0000gn/T/vscode-ssh-askpass-b814f6ec04f38bbaec2d4ceacbd3be8d1447260c.sock"} [11:10:56.214] Spawned 93374 [11:10:56.296] > local-server-1> Running ssh connection command: ssh -v -T -D 49533 -o ConnectTimeout=60 devbox bash [11:10:56.298] > local-server-1> Spawned ssh, pid=93380 [11:10:56.301] stderr> OpenSSH_9.6p1, LibreSSL 3.3.6 [11:10:56.788] stderr> debug1: Server host key: ssh-ed25519 SHA256:bqazz6I8kUfoqbL9tvwFI1AV01uW25hzTa7yVSG0BZA [11:11:00.219] stderr> Authenticated to devbox.panda-cobra.ts.net ([100.72.255.47]:22) using "publickey". [11:11:00.414] > ready: 785087f1c8ab [11:11:00.465] > Linux 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) [11:11:00.465] Platform: linux [11:11:00.516] > /usr/bin/zsh [11:11:00.516] Shell: zsh [11:11:00.516] Creating bash subshell inside "zsh" [11:11:00.575] > bash version: 5.2.15(1)-release [11:11:00.621] > bash version: 5.2.15(1)-release [11:11:00.646] > 785087f1c8ab: running [11:11:00.664] > Found existing installation at /home/rd/.vscode-server...
Starting VS Code CLI... "/home/rd/.vscode-server/code-f1e16e1e6214d7c44d078b1f0607b2388f29d729" command-shell --cli-data-dir "/home/rd/.vscode-server/cli" --on-port --on-host=127.0.0.1 --parent-process-id 18766 &> "/home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log" < /dev/null printenv: [11:11:00.672] > SHELL=/usr/bin/zsh SSH_AUTH_SOCK=/tmp/ssh-XXXXCBgBzV/agent.18763 PWD=/home/rd LOGNAME=rd XDG_SESSION_TYPE=tty MOTD_SHOWN=pam HOME=/home/rd LANG=en_US.UTF-8 VSCODE_AGENT_FOLDER=/home/rd/.vscode-server SSH_CONNECTION=100.127.252.185 49534 100.72.255.47 22 XDG_SESSION_CLASS=user USER=rd SHLVL=2 XDG_SESSION_ID=556 XDG_RUNTIME_DIR=/run/user/1000 ARTIFACTORY_TOKEN=cmVmdGtuOjAxOjE3NDE4MDIxMjM6ZjJwYjRTSXAzQXU3Q01nQzg2bW1jSnJOc3N2 SSH_CLIENT=100.127.252.185 49534 22 PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/go/bin:/home/rd/go/bin:/home/rd/.pulumi/bin DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus OLDPWD=/home/rd _=/usr/bin/printenv [11:11:00.673] > Removing old logfile at /home/rd/.vscode-server/.cli.f1e16e1e6214d7c44d078b1f0607b2388f29d729.log Spawned remote CLI: 18785 [11:11:00.711] > Waiting for server log... [11:11:00.753] > 785087f1c8ab: start SSH_AUTH_SOCK==/tmp/ssh-XXXXCBgBzV/agent.18763== listeningOn==127.0.0.1:45085== osReleaseId==debian== arch==x86_64== vscodeArch==x64== bitness==64== [11:11:00.754] > tmpDir==/run/user/1000== platform==linux== unpackResult==== didLocalDownload==0== downloadTime==== installTime==== serverStartTime==80== execServerToken==11a11a1a-1aa1-11a1-a1a1-aaa1aa1a111a== 785087f1c8ab: end [11:11:00.754] Received install output: SSH_AUTH_SOCK==/tmp/ssh-XXXXCBgBzV/agent.18763== listeningOn==127.0.0.1:45085== osReleaseId==debian== arch==x86_64== vscodeArch==x64== bitness==64==tmpDir==/run/user/1000== platform==linux== unpackResult==== didLocalDownload==0== downloadTime==== installTime==== serverStartTime==80== execServerToken==11a11a1a-1aa1-11a1-a1a1-aaa1aa1a111a==
[11:11:00.755] Remote server is listening on port 45085 [11:11:00.755] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":45085},"osReleaseId":"debian","arch":"x86_64","sshAuthSock":"/tmp/ssh-XXXXCBgBzV/agent.18763","tmpDir":"/run/user/1000","platform":"linux","execServerToken":"11a11a1a-1aa1-11a1-a1a1-aaa1aa1a111a"},"serverStartTime":80,"installUnpackCode":""} [11:11:00.757] Persisting server connection details to /Users/romandodin/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-c4ea1055-f1e16e1e6214d7c44d078b1f0607b2388f29d729-0.113.2024072315-es/data.json [11:11:00.765] Starting forwarding server. local port 49539 -> socksPort 49533 -> remotePort 45085 [11:11:00.765] Forwarding server listening on port 49539 [11:11:00.766] Waiting for ssh tunnel to be ready [11:11:00.770] Tunneled port 45085 to local port 49539 [11:11:00.771] Resolved "ssh-remote+devbox" to "port 49539" [11:11:00.778] Initizing new exec server for ssh-remote+devbox [11:11:00.778] Resolving exec server at port 49539 [11:11:00.802] [Forwarding server port 49539] Got connection 0 [11:11:00.805] [Forwarding server port 49539] Got connection 1 [11:11:00.948] Exec server for ssh-remote+devbox created and cached [11:11:00.949] Updating $SSH_AUTH_SOCK: ln -f -s "/tmp/ssh-XXXXCBgBzV/agent.18763" "/run/user/1000/vscode-ssh-auth-sock-452042854" [11:11:00.950] Using cwd: vscode-remote://ssh-remote%2Bdevbox/ [11:11:00.951] Remote extension host environment: {"SSH_AUTH_SOCK":"/run/user/1000/vscode-ssh-auth-sock-452042854"} [11:11:00.962] ------
[11:11:01.039] [server] Checking /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/log.txt and /home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/pid.txt for a running server... [11:11:01.039] [server] Installing and setting up Visual Studio Code Server... [11:11:01.039] [server] Server setup complete [11:11:01.040] [server] Starting server... [11:11:01.040] [server] Starting server with command... Command { std: "/home/rd/.vscode-server/cli/servers/Stable-f1e16e1e6214d7c44d078b1f0607b2388f29d729/server/bin/code-server" "--connection-token=remotessh" "--accept-server-license-terms" "--start-server" "--enable-remote-auto-shutdown" "--socket-path=/tmp/code-4efab202-3453-4bda-a6c0-6e076f64ceaa", kill_on_drop: false } [11:11:01.116] [server] * [11:11:01.117] [server] * Visual Studio Code Server [11:11:01.117] [server] * [11:11:01.117] [server] * By using the software, you agree to [11:11:01.117] [server] * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and [11:11:01.117] [server] * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement). [11:11:01.117] [server] * [11:11:01.121] [server] Server bound to /tmp/code-4efab202-3453-4bda-a6c0-6e076f64ceaa [11:11:01.121] [server] Extension host agent listening on /tmp/code-4efab202-3453-4bda-a6c0-6e076f64ceaa [11:11:01.122] [server] parsed location: "/tmp/code-4efab202-3453-4bda-a6c0-6e076f64ceaa" [11:11:01.122] [server] [11:11:01.122] [server] Server started [11:11:01.314] [server] [12:11:01] [11:11:01.314] [server] [11:11:01.314] [server] [11:11:01.314] [server] [11:11:01.373] [server] [12:11:01] Extension host agent started. [11:11:01.531] [server] [12:11:01] [
][22521e9f][ManagementConnection] New connection established. [11:11:01.540] [server] [12:11:01] [ ][fa6eaa8b][ExtensionHostConnection] New connection established. [11:11:01.583] [server] [12:11:01] Deleted uninstalled extension from disk charliermarsh.ruff /home/rd/.vscode-server/extensions/charliermarsh.ruff-2024.34.0-linux-x64 [11:11:01.583] [server] [12:11:01] Deleted uninstalled extension from disk github.copilot /home/rd/.vscode-server/extensions/github.copilot-1.214.0 [11:11:03.605] [server] [12:11:03] [ ][fa6eaa8b][ExtensionHostConnection] <19365> Launched Extension Host Process. [11:11:05.848] "Update SSH_AUTH_SOCK" terminal command done [11:11:06.999] Opening exec server for ssh-remote+devbox [11:11:07.016] Opening exec server for ssh-remote+devbox [11:11:07.071] Verified and reusing cached exec server for ssh-remote+devbox [11:11:07.110] Verified and reusing cached exec server for ssh-remote+devbox [11:11:07.189] Opening exec server for ssh-remote+devbox [11:11:07.237] Verified and reusing cached exec server for ssh-remote+devbox
I will keep looking at it through the day.
Thanks, so it sounds like (in "ADD2") things work for you once Enable Agent Forwarding
is enabled?
@connor4312 yes, I've been monitoring it today and it worked without hiccups.
Once I got these ⚠️ icons next to my integrated terminal tabs, and this is when the links broke, but just reopening the terminal fixed it
Thank you and the team for the fixes!
Great, thanks for confirming!
marking as verified bc of the comment from @hellt
Installing v0.113.2024072315 in the stable VS Code also fixed this issue for me.
Seems to work for me too with v0.113.2024072315
- I enabled the "Use Exec Server" option again and I haven't run into stale SSH_AUTH_SOCK so far.
Thanks for fixing this! 🙌