microsoft
Add app-specific configuration for DefTerm handoff
In chatting with @awakecoding on Twitter, I realized that we may be able to add support for application-specific handoff configuration without servicing Windows.
When OpenConsole receives a handoff request in ConsoleEstablishHandoff...
https://github.com/microsoft/terminal/blob/7e47f6aab96b0f2d01309bc05cebe1c57b44a127/src/host/srvinit.cpp#L404-L424
... there's a decision point after it's read the CONNECT packet but before it yeets a PTY connection into Terminal ...
https://github.com/microsoft/terminal/blob/7e47f6aab96b0f2d01309bc05cebe1c57b44a127/src/host/srvinit.cpp#L566-L567
... where we can figure out which application is connecting and look up settings in the old conhost way using RegistrySerialization (et al) (HKCU\Console\C:_Path_To_Whatever.exe or reading the LNK file).
Returning an error from ConsoleEstablishHandoff (or earlier, in CConsoleConnection) will trigger conhost's recovery/fallback mode that hosts console applications locally.
That should enable us to make a late-binding decision in OpenConsole (part of the Terminal package) on whether to continue with handoff.
Risks
This works on the assumption that we can read the same connect packet multiple times; I believe we can, as conhost already read it to figure out wShowWindow.
This will "waste" a megabyte of memory and a process creation on startup as OpenConsole must load and, if it determines not to hand off, exit before the client application gets hooked up to the original conhost.
Thoughts
Interesting thought exercise: OpenConsole could instead choose to host the connection itself. It would provide the same guarantees as conhost (one top-level window, owned by the process) while offering all of the console API, VT and behavioral updates we've made in the past few years that haven't yet trickled out in Windows updates. Folks who have been on the team for a while will recognize this as the effort I used to call "conhost coast to coast"¹.
¹ (we originally planned on moving just conhost to a package and updating it via the store, before Terminal was approved!)
First things first, I would like to put things in context, and why an application-level override is very important: many programs (like ours) launch PowerShell and reparent it into a parent program. Windows Terminal breaks window reparenting, and even if it did support it properly, we would still need the option to control which console host to use from Remote Desktop Manager, instead of relying on global system defaults. The technique for window reparenting is simple: we launch the shell, wait for it to become visible, and then use Process.MainWindowHandle to set the parent window.
function Test-IsWindowsTerminalDefault() {
$DelegationConsole = (Get-ItemProperty 'HKCU:\Console\%%Startup' -Name 'DelegationConsole' -ErrorAction SilentlyContinue).DelegationConsole
$DelegationTerminal = (Get-ItemProperty 'HKCU:\Console\%%Startup' -Name 'DelegationTerminal' -ErrorAction SilentlyContinue).DelegationTerminal
if ([string]::IsNullOrEmpty($DelegationConsole) -or [string]::IsNullOrEmpty($DelegationTerminal)) {
return $False
}
if (($DelegationConsole -eq '{00000000-0000-0000-0000-000000000000}') -or ($DelegationTerminal -eq '{00000000-0000-0000-0000-000000000000}')) {
return $False
}
$True
}
Test-IsWindowsTerminalDefault
$ConsoleProcess = Start-Process -FilePath 'pwsh.exe' -PassThru
$ConsoleProcessId = $ConsoleProcess.Id
$ConsoleProcess.Id
$ConsoleProcess.MainWindowHandle
And here is a screenshot where I have set the Windows Console Host as the default, launched PowerShell, then set Windows Terminal as the default, and launched PowerShell again:
You can see that with the Windows Console Host, ConsoleProcess.MainWindowHandle is non-zero, but with Windows Terminal, ConsoleProcess.MainWindowHandle is zero. In both cases, the PID of the new shell matches the process we have just launched.
The ideal solution, in my mind, would be to allow overriding the terminal through an environment variable, such that existing programs that wish to force the old console host can be updated with fairly minimal changes. The only workaround that works today requires significant code changes, and has many drawbacks: wrapping the console process inside conhost.exe. Here is another PowerShell code snippet to demonstrate the effect of wrapping pwsh.exe inside conhost.exe to force the old console host when Windows Terminal is set as the default:
$EmbeddedProcess = Start-Process -FilePath 'conhost.exe' -ArgumentList @('pwsh.exe') -PassThru
$EmbeddedProcessId = $EmbeddedProcess.Id
$EmbeddedProcess.Id
$EmbeddedProcess.MainWindowHandle
$CapturedProcessId = Get-CimInstance -ClassName Win32_Process -Filter "ParentProcessId = $EmbeddedProcessId" | Select-Object -ExpandProperty ProcessId -First 1
$CapturedProcess = Get-Process -Id $CapturedProcessId
$CapturedProcess.Id
$CapturedProcess.MainWindowHandle
And here is a screenshot showing the end result:
Wrapping pwsh.exe inside conhost.exe also breaks window reparenting, because the real process you want to check is the child process (pwsh.exe) which has the usable Process.MainWindowHandle value. Finding the child process and modifying the wrapper code logic to handle both "direct" and "indirect" processes is the kind of change that can easily introduce new bugs. Moreover, wrapping processes to run inside conhost.exe breaks certain things, like the default app icon and title (it says conhost.exe instead of pwsh.exe, and the icon is the one from the command-prompt, not PowerShell). The title and icon don't matter when reparenting, but they do matter when launching external shells that aren't embedded (external mode in Remote Desktop Manager).
Again, the ideal solution would be to allow overriding system defaults for the console host through an environment variable. I'm not sure if we want just an environment variable to force the old console host, or also one to force Windows Terminal if the old console host is the default. I would be happy with something like "CONHOST_FORCE_LEGACY=1"
Any update on this? I'll be working on patching Windows Terminal for a custom Devolutions distribution which will later ship with Remote Desktop Manager, so I can try a few things starting next week.
I just spent 30 minutes explaining to one of our developers the convoluted logic in Remote Desktop Manager that detects which console host we intend to use, which one we can use, and the code paths that should use the conhost.exe hack or not, and when. It's really messy, everybody's confused, can't we just fix this by using an environment variable to force the old console host?