vscode-remote-release [Remote-SSH Bug]: Cannot reconnect to ssh agent in the same terminal after reload window

Is there an existing issue for this bug?

[X] I have searched the existing issues

Debugging Steps

[X] I have followed these debugging steps
[X] I have tried both values of the remote.SSH.useLocalServer setting
[X] My SSH command works from a local terminal
[X] I can connect using the Remote-SSH extension on VS Code when all other VS Code extensions are disabled

->

remote.SSH.useLocalServer=false does not have this issue.

Versions

VS Code Version: 1.74.3
Remote-SSH Version: v0.94.0
Remote OS Version: ubuntu 20.04

Expected Behavior

Cannot reconnect to ssh agent in the same terminal after reload window.

❯ ssh-add -L
Error connecting to agent: No such file or directory

Steps To Reproduce

Set remote.SSH.useLocalServer=true
Connect to an SSH host

Open a terminal and check SSH_AUTH_SOCK:

 ❯ ls -L $SSH_AUTH_SOCK
/run/user/1001/vscode-ssh-auth-sock-334051596

Reload Window

In the same terminal, check SSH_AUTH_SOCK:

❯ ls -L $SSH_AUTH_SOCK
ls: cannot access '/run/user/1001/vscode-ssh-auth-sock-334051596': No such file or directory

Because SSH_AUTH_SOCK is pointing at a blackhole, the ssh agent will fail to find any keys:
```
❯ ssh-add -L
Error connecting to agent: No such file or directory
```

If you open a new terminal you'll get a new SSH_AUTH_SOCK value which works:

❯ ls -L $SSH_AUTH_SOCK        
/run/user/1001/vscode-ssh-auth-sock-961955974

Remote-SSH Log

Remote-SSH Log when reloading the window

[12:37:07.828] Log Level: 2
[12:37:07.831] [email protected]
[12:37:07.831] darwin arm64
[12:37:07.834] SSH Resolver called for "ssh-remote+server-compute", attempt 1
[12:37:07.835] "remote.SSH.useLocalServer": true
[12:37:07.835] "remote.SSH.path": undefined
[12:37:07.835] "remote.SSH.configFile": undefined
[12:37:07.835] "remote.SSH.useFlock": true
[12:37:07.835] "remote.SSH.lockfilesInTmp": false
[12:37:07.835] "remote.SSH.localServerDownload": auto
[12:37:07.835] "remote.SSH.remoteServerListenOnSocket": false
[12:37:07.835] "remote.SSH.showLoginTerminal": false
[12:37:07.835] "remote.SSH.defaultExtensions": ["ms-python.python","ms-python.vscode-pylance"]
[12:37:07.835] "remote.SSH.loglevel": 2
[12:37:07.836] "remote.SSH.maxReconnectionAttempts": 2
[12:37:07.836] "remote.SSH.enableDynamicForwarding": true
[12:37:07.836] "remote.SSH.enableRemoteCommand": false
[12:37:07.836] "remote.SSH.serverPickPortsFromRange": {}
[12:37:07.836] "remote.SSH.serverInstallPath": {}
[12:37:07.840] SSH Resolver called for host: server-compute
[12:37:07.840] Setting up SSH remote "server-compute"
[12:37:07.841] Acquiring local install lock: /var/folders/5y/hhvf46cx393_j75q4gm7tvzw0000gq/T/vscode-remote-ssh-0efbde2a-install.lock
[12:37:07.843] Looking for existing server data file at /Users/tekumara/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-0efbde2a-97dec172d3256f8ca4bfb2143f3f76b503ca0534-0.94.0/data.json
[12:37:07.843] Using commit id "97dec172d3256f8ca4bfb2143f3f76b503ca0534" and quality "stable" for server
[12:37:07.845] Install and start server if needed
[12:37:07.847] PATH: /Users/tekumara/.cargo/bin:/Users/tekumara/Library/Caches/fnm_multishells/61649_1673852755642/bin:/Users/tekumara/.pyenv/shims:/Users/tekumara/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/tekumara/.local/bin:/usr/local/opt/go/libexec/bin:/Users/tekumara/go/bin:/opt/homebrew/opt/fzf/bin
[12:37:07.847] Checking ssh with "ssh -V"
[12:37:07.856] > OpenSSH_9.0p1, LibreSSL 3.3.6

[12:37:07.858] askpass server listening on /var/folders/5y/hhvf46cx393_j75q4gm7tvzw0000gq/T/vscode-ssh-askpass-992ad2b5af562fbb2de23937eec5a2d17c93a4f4.sock
[12:37:07.858] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/5y/hhvf46cx393_j75q4gm7tvzw0000gq/T/vscode-ssh-askpass-a27aaebd0e8c48028140920c636ee60413113036.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","64266","-o","ConnectTimeout=60","server-compute"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/tekumara/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-0efbde2a-97dec172d3256f8ca4bfb2143f3f76b503ca0534-0.94.0/data.json"}
[12:37:07.858] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.QSP5RMozFZ/Listeners","SHELL":"/bin/zsh","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/tekumara/.vscode/extensions/ms-vscode-remote.remote-ssh-0.94.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper.app/Contents/MacOS/Code Helper","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"--ms-enable-electron-run-as-node","VSCODE_SSH_ASKPASS_MAIN":"/Users/tekumara/.vscode/extensions/ms-vscode-remote.remote-ssh-0.94.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5y/hhvf46cx393_j75q4gm7tvzw0000gq/T/vscode-ssh-askpass-992ad2b5af562fbb2de23937eec5a2d17c93a4f4.sock"}
[12:37:07.859] Spawned 49701
[12:37:07.926] > local-server-1> Running ssh connection command: "-v -T -D 64266 -o ConnectTimeout=60 server-compute"
[12:37:07.928] > local-server-1> Spawned ssh, pid=49710
[12:37:07.934] stderr> OpenSSH_9.0p1, LibreSSL 3.3.6
[12:37:10.632] stderr> Connecting to instance i-0514fc429b6c3e1f0
[12:37:14.939] stderr> debug1: Server host key: ssh-ed25519 SHA256:dLWdQIJwrN1tOBe1yuRTWtBohf4QnfW2QrnQyvwH++g
[12:37:16.715] stderr> Authenticated to server-compute (via proxy) using "publickey".
[12:37:17.645] > 
> 
>   System information as of Tue Jan 17 01:37:51 UTC 2023
> 
> 
> 111 updates can be applied immediately.
> 2 of these updates are standard security updates.
> To see these additional updates run: apt list --upgradable
> 
> 
> The list of available updates is more than a week old.
> To check for new updates run: sudo apt update
> New release '22.04.1 LTS' available.
> Run 'do-release-upgrade' to upgrade to it.
> 
> 
> *** System restart required ***
[12:37:17.888] > ready: e2fb0693676b
[12:37:18.139] > Linux 5.15.0-1022-aws #26~20.04.1-Ubuntu SMP Sat Oct 15 03:22:07 UTC 2022
[12:37:18.139] Platform: linux
[12:37:18.878] > /usr/bin/zsh
[12:37:18.879] Shell: zsh
[12:37:18.879] Creating bash subshell inside "zsh"
[12:37:19.132] > bash version:  5.0.17(1)-release
[12:37:19.353] > bash version:  5.0.17(1)-release
[12:37:19.353] > bash version:  5.0.17(1)-release
[12:37:19.357] > bash version:  5.0.17(1)-release
> bash version:  5.0.17(1)-release
[12:37:19.616] > e2fb0693676b: running
[12:37:19.616] > Acquiring lock on /home/compute/.vscode-server/bin/97dec172d3256f8ca4bfb2143f3f76b503ca0534/vscode-remote-lock.compute.97dec172d3256f8ca4bfb2143f3f76b503ca0534
> Found existing installation at /home/compute/.vscode-server/bin/97dec172d3256f8ca4bfb2143f3f76b503ca0534...
> Checking /home/compute/.vscode-server/.97dec172d3256f8ca4bfb2143f3f76b503ca0534.log and /home/compute/.vscode-server/.97dec172d3256f8ca4bfb2143f3f76b503ca0534.pid for a running server
> Looking for server with pid: 4017170
> Found running server...
> Installing extensions...
[12:37:19.907] > Installing extensions...
[12:37:20.201] > Extension 'ms-python.python' v2022.20.2 is already installed. Use '--force' option to update to latest version or provide '@<version>' to install a specific version, for example: '[email protected]'.
[12:37:20.203] > Extension 'ms-python.vscode-pylance' v2023.1.20 is already installed. Use '--force' option to update to latest version or provide '@<version>' to install a specific version, for example: '[email protected]'.
[12:37:20.213] >  
> *
> * Visual Studio Code Server
> *
[12:37:20.214] > * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
[12:37:20.225] > Checking server status on port 34323 with wget
[12:37:20.437] > e2fb0693676b: start
> SSH_AUTH_SOCK==/tmp/ssh-UyvOFsodIW/agent.4018972==
> DISPLAY====
> webUiAccessToken====
> listeningOn==34323==
> osReleaseId==ubuntu==
> arch==x86_64==
> tmpDir==/run/user/1001==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
[12:37:20.437] > extInstallTime==731==
> serverStartTime====
> connectionToken==1a111aaa-11a1-1a11-11aa-1aa1a1a1aa11==
> e2fb0693676b: end
[12:37:20.437] Received install output: 
SSH_AUTH_SOCK==/tmp/ssh-UyvOFsodIW/agent.4018972==
DISPLAY====
webUiAccessToken====
listeningOn==34323==
osReleaseId==ubuntu==
arch==x86_64==
tmpDir==/run/user/1001==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====extInstallTime==731==
serverStartTime====
connectionToken==1a111aaa-11a1-1a11-11aa-1aa1a1a1aa11==

[12:37:20.438] Remote server is listening on 34323
[12:37:20.438] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":34323},"osReleaseId":"ubuntu","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"/tmp/ssh-UyvOFsodIW/agent.4018972","display":"","tmpDir":"/run/user/1001","platform":"linux","connectionToken":"1a111aaa-11a1-1a11-11aa-1aa1a1a1aa11"},"extInstallTime":731,"installUnpackCode":""}
[12:37:20.439] Persisting server connection details to /Users/tekumara/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-0efbde2a-97dec172d3256f8ca4bfb2143f3f76b503ca0534-0.94.0/data.json
[12:37:20.442] Starting forwarding server. localPort 64288 -> socksPort 64266 -> remotePort 34323
[12:37:20.443] Forwarding server listening on 64288
[12:37:20.443] Waiting for ssh tunnel to be ready
[12:37:20.443] Tunneled 34323 to local port 64288
[12:37:20.443] Resolved "ssh-remote+server-compute" to "127.0.0.1:64288"
[12:37:20.444] Updating $SSH_AUTH_SOCK: ln -f -s "/tmp/ssh-UyvOFsodIW/agent.4018972" "/run/user/1001/vscode-ssh-auth-sock-961955974"
[12:37:20.444] Using cwd: vscode-remote://ssh-remote%2Bserver-compute/
[12:37:20.444] Remote extension host environment: {"SSH_AUTH_SOCK":"/run/user/1001/vscode-ssh-auth-sock-961955974"}
[12:37:20.446] [Forwarding server 64288] Got connection 0
[12:37:20.452] ------




[12:37:20.470] [Forwarding server 64288] Got connection 1
[12:37:20.994] [Forwarding server 64288] Got connection 2
[12:37:24.869] [Forwarding server 64288] Got connection 3
[12:37:24.869] [Forwarding server 64288] Got connection 4
[12:37:28.331] "Update SSH_AUTH_SOCK" terminal command done

Anything else?

This issue also occurs when the ssh connection has been lost, but vscode hasn't quit, and you reconnect by reloading the window.

Jan 17 '23 01:01 tekumara

sorry for the delay- I'm going to attempt to repro. Thanks

Jan 23 '23 18:01 eleanorjboyd

Hello! I think this is related to an ongoing issue we have with useLocalServer where it is not actually reusing the local server correctly. We are aware of this issue and plan to get it fixed soon so hopefully this will be the fix. You can follow along here https://github.com/microsoft/vscode-remote-release/issues/6227 and I will keep this open while the issue is not yet addressed.

Jan 31 '23 22:01 eleanorjboyd

Anyone coming here wanting a quick fix, I've done the following which works on my machine (vscode on mac, running remote on linux)

I've added the following line to my remote ~/.zshrc

[[ -S $(readlink $SSH_AUTH_SOCK) ]] || ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK

What this does is, check if the symlink in $SSH_AUTH_SOCK is pointing to a socket, if not it takes the last created agent socket and links to it.

Validate with ssh-add -L

Just my 2c

Edit: I noticed that on a regular login $SSH_AUTH_SOCK points to a socket and not a symlink. Therefore the following works better. Also, this takes into account that $SSH_AUTH_SOCK just points to a file that's just not there

# vscode SSH_AUTH_SOCK fix
(test -L $SSH_AUTH_SOCK && (
    test -S $(readlink $SSH_AUTH_SOCK) && \
        ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK
)) || ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK

Feb 14 '23 14:02 screencomuser

Anyone coming here wanting a quick fix, I've done the following which works on my machine (vscode on mac, running remote on linux)

I've added the following line to my remote ~/.zshrc

[[ -S $(readlink $SSH_AUTH_SOCK) ]] || ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK

What this does is, check if the symlink in $SSH_AUTH_SOCK is pointing to a socket, if not it takes the last created agent socket and links to it.

Validate with ssh-add -L

Just my 2c

Edit: I noticed that on a regular login $SSH_AUTH_SOCK points to a socket and not a symlink. Therefore the following works better. Also, this takes into account that $SSH_AUTH_SOCK just points to a file that's just not there
# vscode SSH_AUTH_SOCK fix
(test -L $SSH_AUTH_SOCK && (
    test -S $(readlink $SSH_AUTH_SOCK) && \
        ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK
)) || ln -sf $(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) $SSH_AUTH_SOCK

i tried the quickfix, but this didn't work out of the box, because the symlink could not be created (i didnt look into the matter deeper, tho)

instead, i added to my .bashrc the following: [[ -S $(readlink $SSH_AUTH_SOCK) ]] || export SSH_AUTH_SOCK=$(ls -1rtd /tmp/ssh-*/* | tail -1 | xargs) this basically does the same, except it does not create a symlink to $SSH_AUTH_SOCK, but instead simply overwrites the environment variable with the most recent created SSH socket

Mar 21 '24 11:03 apfreelancing

My solution

#!/bin/bash

SSH_EXISTING_SOCK=$(find /tmp -maxdepth 1 -type d -name 'ssh-*' -user $USER | xargs -I {} find {} -type s)

if [ -n "$SSH_AUTH_SOCK" ]; then
    mkdir -p $(dirname $SSH_AUTH_SOCK)
    ln -sf $SSH_EXISTING_SOCK $SSH_AUTH_SOCK
fi

Jun 24 '24 09:06 thomasmebes

I have followed the reproduction steps and can no longer repo. We made some changes to SSH_AUTH_SOCK in the second half of this year that likely helped this scenario.

I ask that anyone that can repro please create a new issue with your logs so that I can debug - thank you!

Dec 17 '24 19:12 joshspicer

vscode-remote-release vscode-remote-release copied to clipboard

[Remote-SSH Bug]: Cannot reconnect to ssh agent in the same terminal after reload window

Is there an existing issue for this bug?

Debugging Steps

->

Versions

Expected Behavior

Steps To Reproduce

Remote-SSH Log

Anything else?

vscode-remote-release
vscode-remote-release copied to clipboard