[Remote-SSH Bug]: Failed connections with some embedded remote devices

[the-moog]

Is there an existing issue for this bug?

• [X] I have searched the existing issues

Required Troubleshooting Steps

• [X] I have followed these troubleshooting steps
• [X] I have tried both values of the remote.SSH.useLocalServer setting

Connect Locally

It doesn’t connect successfully (same error message)

->

I tried several combinations of configurations, all failed with varying error messages and clearly the root cause was the same in each case.

Expected Behavior

Connection - as it used to

Steps To Reproduce

I tried to re-visit an embedded project that I started a few years ago but have not touched for a year or so. This project is for the QNAP NAS, though for this bug that fact is mostly irrelevant as from searching I can see that the same bug affects attempts to run VSCode/remote/ssh on embedded systems that lack all the utilities found in less resource limited systems.

My project is a framework that uses simple recipes to allow local compilation of various applications missing from the stock (and rather outdated) QNAP OS called QTS. So this is a bit chicken and egg as if I could get it working I'd have a platform that would help me fix it!!

I think that I last used this about 18 months ago, probably on QTS v4.x rather than 5.x. Back then I was able to use VSCode as an IDE for this project, so was surprised when I found it no longer works.

Investigations showed this is due to the fact that the QNAP uses busybox and that it also lacks two other common commands used by the VSCode remote installer but that are missing on QTS.

This causes multiple issues related to VSCode Remote/SSH install/upgrade on connection:

1: The installer ssh connection appears to be configured to bypass (or delay) the BASH login scripts, both interactive and non-interactive types. For my, currently though still trying, any attempt to monkey-patch this is thwarted. This did not used to be the case so VSCode's installer changed at some point. A modified environment required for remote dev used to work.

2: The vscode-cli It is calling the ps command with the p (or -p ??) switch to get the process id for the remote server. However busybox, a sort of Swiss Army Knife toolbox containing many (borked) standard Unix CLI apps and which is commonly used on embedded systems (for outdated reasons IMO) does not understand the p switch, (true at least the busybox on QNAP dunno if this is endemic or a compile time config). As this switch is not understood, this results in a snowball of errors:

3: A couple other commands are also missing in the QTS Operating System.

See the attached log file.

...

On that consider this (completely off topic) aside: I note some typos in the error log. I only mention this as this can cause problems should people search for the error terms online and spell them correctly. Initizing is not really a word, (ref OED) errored is not really a word, (ref OED) and is superfluous even if it were. 'Canceled` is correct - in the US, but as I am British (and yes, I am now being rather pedantic) IMO it should have two l's

...

Back on topic:

So, because it fails to find the PID of the server using ps it can't report the listeningOn port, and because of that it can't make a connection to that port so it fails. I believe -p is a POXIX standard switch so should be accepted - (busybox has a lot to answer for)

The same happens if other connection config mechanisms are used as it seems the ps command is always used, as is getconf and printenv.

The other missing commands are probably less of an issue, namely getconf and printenv. I think printenv could simply be replaced with env, which is perhaps more universal, or maybe just expand the required env vars and handle the cases where they are missing. What envs are needed?

I think the absence of getconf is an upshot of the ancient Kernel in QNAP (ancient AND obsolete - why QNAP won't update is causing many other issues). I checked and getconf is in POSIX, so it's annoying it's missing and hence not really a VSCode bug, rather one in QNAP - though I won't hold my breath until QNAP fix this. I understand if avoiding it causes more trouble than using it.

I tried writing wrappers for the missing commands, but as it seems the remote users environment is not picked up by VSCode's initial login any attempt to redirect or alias commands is prevented. The OS on QNAP is on a read-only file system so I can't simply add them and ssh is locked down to a minimal set of functions.

Some hacky work-rounds I've yet to try (pending a better fix):

• Supplying a PATH via SSH and/or other push tricks that ssh allows. Fairly sure that QNAPs sshd will disallow this so have not tried yet. The stock ssdh_config is very restrictive.
• Somehow getting an overlay file system to replace some of the commands with wrappers or locally compiled versions. This is going to be very complex, and likely to break more things, so also not tried.
• Running an ssh server on another port with less restrictions. How to tell VSCode that ssh is on a non-standard port during install?

Some suggestions for VSCode that would help fix this:

• Provide a mechanism to enable the normal user login environment to be present before the install starts.
• If practical, replace the absolute dependencies on ps, printenv and getconf and/or include multiple mechanisms and a way to probe one that works for any given system.

Remote-SSH Log

Remote-SSH Log

Example error log (slightly abbreviated/obfuscated):

> 
[16:06:58.528] > dcc7e94f8a14: running
[16:06:58.541] > bash: line 108: getconf: command not found
> Found existing installation at /share/homes/user/.vscode-server...
> Starting VS Code CLI... "/share/homes/user/.vscode-server/code-fee1edb8d6d.......24b9" command-shell --cli-data-dir "/share/homes/user/.vscode-server/cli" --on-port --on-host=127.0.0.1 --parent-process-id 18887 &> "/share/homes/user/.vscode-server/.cli.fee1......4b9.log" < /dev/null
> printenv:
> bash: line 79: printenv: command not found
> Removing old logfile at /share/homes/user/.vscode-server/.cli.fee1....4b9.log
> Spawned remote CLI: 18913
> ps: invalid option -- 'p'
> BusyBox v1.24.1 (2024-08-17 02:56:18 CST) multi-call binary.
> 
> Usage: ps [-o COL1,COL2=HEADER] [-T]
> 
> Show list of processes
[16:06:58.565] > 
> 
>         -o COL1,COL2=HEADER     Select columns for display
>         -T                      Show threads
> 
> QNAP Options:
>         --columns N
> 
> Exec server process not found
> dcc7e94f8a14: start
> DISPLAY====
> listeningOn====
> osReleaseId==qts==
> arch==x86_64==
> vscodeArch==x64==
> bitness====
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> serverStartTime==9==
> execServerToken==1aa11aaa-1a11-111a-1aa1-111111aa1aa1==
> dcc7e94f8a14: end
[16:06:58.565] Received install output: 
DISPLAY====
listeningOn====
osReleaseId==qts==
arch==x86_64==
vscodeArch==x64==
bitness====
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
serverStartTime==9==
execServerToken==1aa11aaa-1a11-111a-1aa1-111111aa1aa1==

[16:06:58.565] Failed to parse remote port from server output
[16:06:58.566] Resolver error: Error: 
	at m.Create (c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:501230)
	at t.handleInstallOutput (c:\Users\user\.vscode\extensions\ms-vscode-remote.remotessh-0.113.2024072315\out\extension.js:2:498552)
	at t.tryInstall (c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:623217)
	at async c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:582788
	at async t.withShowDetailsEvent (c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:586054)
	at async k (c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:579675)
	at async t.resolve (c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:583465)
	at async c:\Users\user\.vscode\extensions\ms-vscode-remote.remote-ssh-0.113.2024072315\out\extension.js:2:851047
[16:06:58.568] ------

[16:06:58.854] > 
[16:06:58.861] Opening exec server for ssh-remote+nas6
[16:06:58.865] Initizing new exec server for ssh-remote+nas6
[16:06:58.882] Using commit id "fee1...924b9" and quality "stable" for server
[16:06:58.883] Install and start server if needed
[16:06:58.886] Opening exec server for ssh-remote+nas6
[16:07:08.366] getPlatformForHost was canceled
[16:07:08.366] Exec server for ssh-remote+nas6 failed: Error: Connecting was canceled
[16:07:08.366] Existing exec server for ssh-remote+nas6 errored (Error: Connecting was canceled)
[16:07:08.367] Initizing new exec server for ssh-remote+nas6
[16:07:08.367] Using commit id "fee1e...924b9" and quality "stable" for server
[16:07:08.368] Error opening exec server for ssh-remote+nas6: Error: Connecting was canceled
[16:07:08.368] Install and start server if needed
[16:08:23.401] getPlatformForHost was canceled
[16:08:23.402] Exec server for ssh-remote+nas6 failed: Error: Connecting was canceled
[16:08:23.402] Error opening exec server for ssh-remote+nas6: Error: Connecting was canceled

Anything else?

No response