Enhance shell autocompletion with a cool new user interface and shell completion protocol

[anki-code]

Hello! Thank you for the new interesting project!

I just want to let you know about 🚀 Upterm — really great proof of concept but it stopped because maintainer was gone. This terminal looks like 21st century terminal. Very sad that it isn't supported.

[zadjii-msft]

This looks really cool, though this looks like something the shell would have to work with the terminal to achieve.

How was Upterm providing those autocomplete suggestions? Did they have their own shell they were using, or were they somehow pulling the auto-completions from bash/zsh/git somehow?

Presuming the shell provided a list of autocompletion suggestions to the terminal, drawing the UI shouldn't be that hard. Cmd would obviously never be able to support this, but powershell core sure could. I'd be curious what kind of perf there would be for something like that, with the shell emitting a list of auto-completion suggestions after some delay/on every character typed.

Doing the collapsible json thing might be quite a bit harder however :P that definitely seems like they had a custom rolled cat that worked directly with the terminal.

[huoyaoyuan]

I think you can get inspired by Visual Studio's C# Interactive and PowerShell ISE. PowerShell already has a mechanism to collect completions, which is used by dotnet.exe. And also, don't forget the great Language Server Protocol by Visual Studio.

[n3wt0n]

That would be really cool, or something like the iTerm 2 autocompletion on Mac. (cmd + ;)

This works for both files/folder and commands

[remkop]

@zadjii-msft

Cmd would obviously never be able to support this, but powershell core sure could.

Cmd could certainly achieve bash-like completion: clink enhances Cmd with tab completion, which users can customize with simple lua scripts.

While not as fancy looking as the screenshots above, it would be hugely valuable if Cmd would support this out of the box.

(More detail: see docs and pre-built completions for common tools.)

[zadjii-msft]

I don't disagree, projects like clink and yori are great and I love them. It's just that we really can't accept any changes to cmd.exe safely 😕 This doc covers some of the reasons why.

[remkop]

@zadjii-msft Thank you for the clarification. Makes sense and made me realize what a great job you and your team are doing. Keep it up! (Also thanks for pointing me to yori, I did not know about that project.)

[just1a-person]

Wish we had the Terminal of 2005:

[DHowett]

You know, apart from the "graphical" bit, this sounds like PowerShell. It accepts C# (procedural and object-oriented), you can write prompt plugins, you can stream files (and objects!), and it supports multiple hosting interfaces such as the PowerShell ISE (which is graphical!)

[just1a-person]

There is this new terminal called Warp: https://twitter.com/zachlloydtweets/status/1415343353164599299

I wish Windows Terminal was that radical and not just try to match existing Unix terminals' functionality (which it already has most of)

[orcmid]

There is this new terminal called Warp: https://twitter.com/zachlloydtweets/status/1415343353164599299

I wish Windows Terminal was that radical and not just try to match existing Unix terminals' functionality (which it already has most of)

I think there is a confusion of shell versus terminal and how apps are wired in. So Warp seems more integrated and purpose-built. All the questions about how one uses a terminal are about session shell use. They focus on CLI.

Since there was such a thing as stealth mode, one must presume this is to be a commercial offering. There is no hint that the code is even in the open (although there is a statement about working in public). "Stable" releases are also ratcheting pretty quickly according to the Discord. warp.dev has more. There's a nice presentation of How Warp Works. I have not found any statement about licensing or commercial use, although having forked Alacritty might be relevant, depending on how the permissive Apache License is dealt with.

[just1a-person]

I think the PowerShell team (or PSReadLine) and the Windows Terminal team can work together to make something like this happen.

[IgnusG]

There’s a plug-in for the Mac terminal that offers this functionality too

https://github.com/withfig/autocomplete

[IgnusG]

There is this new terminal called Warp: https://twitter.com/zachlloydtweets/status/1415343353164599299

I wish Windows Terminal was that radical and not just try to match existing Unix terminals' functionality (which it already has most of)

This looks amazing. Hope that MS Terminal can match these features at some point. The visual blocks and auto complete/history view would be awesome.

Maybe if the plug-in interface is permissive enough it could be shipped as part of plug-ins that work together with the shell.

[ntindle]

If I wanted to work on this, is it possible or would it be better as an extension (and thus need the extension support to be completed)?

[zadjii-msft]

@ntindle I suppose we wouldn't need extension support to add this to the Terminal first, if we could figure out a halfway decent way of building this into the Terminal. I guess I'd be curious to hear how you planned on implementing this. From my POV there's a couple ways we could go about this, each with upsides and drawbacks, so I'm curious what you had in mind ☺️

[ntindle]

Hello,

To be honest, I am wildly unfamiliar with the terminal’s code base and overall architecture. I am certain I am not the best programmer for the job, but I am interested in using the final result, even if I have to build it. That said, here we go…

My initial thoughts would be that this would have a multi stage implementation at different layers with the shell and outer terminal both needing work.

The terminal would need to be able to render “panes” (I believe pane means something specific in the context of the terminal but for now I’m just referring to a generalized UI container) on top of the shell when it receives a trigger from terminal due to time since last keypress (or elsewhere with other triggers).

The profile for each shell would use the shells respective autocomplete tooling to pull the autocomplete actions out and provide them to the terminal. This would probably be one of the simplest ways to support a wide variety of actions without having to build the auto complete engine from scratch.

This interaction would be similar to how VS Code handles language servers and auto complete in that the terminal would both decide when to show the “pane” and query the shell for the appropriate next characters.

A different approach from the completion side would be to build the equivalent of language servers that provide intelligent completions using context from the shell. I think this one would be much more powerful and popular but significantly more complex to support and implement. I believe this is how the Fig example above works.

I would greatly appreciate any advice or ideas on this, even if it is to leave it to someone with a better understanding of the terminal’s code base.

[zadjii-msft]

A tiny prototype is in ~dev/migrie/fhl/menu-complete-prototype~ dev/migrie/fhl/vscode-autocomplete-prototype

• I hate the current design for the VT sequence. Performance is terrible. We should be able to just set the whole menu with a single sequence.
• We probably don't need to leave the space for the icon in the menu
• We should get rid of the alphabetical ordering for commands in the autocomplete version of the menu.
• We may want to allow the menu to be modal, s.t. a shell could still key character keypresses, but arrows, tab, enter would go to the menu.
  • We should think hard if this is a shell-specified setting (like, a flag passed in the OSC sequence) or a terminal-side setting.
• We probably don't need an autocomplete menu and a command palette. Or we could certainly create a simpler shared base class.
• We should definitely get rid of the > prompt for "action mode" in the autosuggest menu cause that's obviously dumb.

As a stretch related idea: It would be really cool if the ControlCore saved working directories as it saw them, and then, you could open the auto suggest menu and it's pre-populated with a stack of your recent directories for this pane, as sendInput(cd ${path})` actions.

[zadjii-msft]

Alrighty folks, so, it's discussion time.

I've got a prototype of this out over in #14938. That's absolutely a first-draft of this experience, and there's plenty of room for improvements.

Right now, the "protocol" it is using is an OSC sequence with a blob of JSON that contains a serialization of the output of the PowerShell cmdlet TabExpansion2. This is obviously terrible. It's needlessly verbose, and it's super powershell-specific. Not only that, I suspect this doesn't include all the information we'd want to send, either.

Right now, the payload of that blob looks something like:

[
    {
        "CompletionText": "Microsoft.PowerShell.Management",
        "ListItemText": "Microsoft.PowerShell.Management",
        "ResultType": 8,
        "ToolTip": "Description: \r\nModuleType: Manifest\r\nPath: C:\\program files\\powershell\\7\\Modules\\Microsoft.PowerShell.Management\\Microsoft.PowerShell.Management.psd1"
    },
    {
        "CompletionText": "Microsoft.PowerShell.Utility",
        "ListItemText": "Microsoft.PowerShell.Utility",
        "ResultType": 8,
        "ToolTip": "Description: \r\nModuleType: Manifest\r\nPath: C:\\program files\\powershell\\7\\Modules\\Microsoft.PowerShell.Utility\\Microsoft.PowerShell.Utility.psd1"
    },
    {
        "CompletionText": "PSReadLine",
        "ListItemText": "PSReadLine",
        "ResultType": 8,
        "ToolTip": "Description: Great command line editing in the PowerShell console host\r\nModuleType: Script\r\nPath: C:\\program files\\powershell\\7\\Modules\\PSReadLine\\PSReadLine.psm1"
    }
]

Which is synthensized with a powershell command like:

function Send-Completions {
    $commandLine = ""
    $cursorIndex = 0
    
    [Microsoft.PowerShell.PSConsoleReadLine]::GetBufferState([ref]$commandLine, [ref]$cursorIndex)
    $completionPrefix = $commandLine

    # Get completions
    $result = "`e]633;Completions"
    if ($completionPrefix.Length -gt 0) {
        # Get and send completions
        $completions = TabExpansion2 -inputScript $completionPrefix -cursorColumn $cursorIndex
        if ($null -ne $completions.CompletionMatches) {
            $result += ";$($completions.ReplacementIndex);$($completions.ReplacementLength);$($cursorIndex);"
            $result += $completions.CompletionMatches | ConvertTo-Json -Compress
        }
    }
    $result += "`a"

    Write-Host -NoNewLine $result
}

Now, the point of discussion: How do we want to refine this sequence?

• How do we make this something that pwsh, bash, zsh, fish, nu... anyone really, could emit easily?
• Similarly, the same menu could be used by something like vim. Would it be possible there?
• What's missing from this protocol that these other shells might also want?
• TabExpansion2 isn't even that great as far as PowerShell is concerned - there's more elaborate completion providers that might give us more info
• An OSC seems a little silly for this - would a DCS make more sense? an APC (ew)? What's the actual wire format look like?

A quick /cc to @Tyriar who's driving this on the VsCode terminal side.

I may also want to cc @j4james, @wez, @christianparpart who usually have opinions on this sort of thing. I'd rather get some initial design feedback here before writing a more formal spec for terminal-wg (which has kinda just become informative, not normative anyways)

[wez]

Neat!

The fig.io folks may have some thoughts on this, their thing is bolting similar looking functionality on top of existing terminals through a pty and IME sandwich. https://github.com/withfig/autocomplete in case you haven't seen it. cc: @brendanfalk; not sure how much you're up for sharing/collaborating on this, but I suspect that you probably already have some OSC sequences that are similar in purpose!

May also be worth looping in say, one of the fish folks, to get their thoughts from the client side. I actually don't know how their line editor works so I don't know if they would be a prime candidate for this.

[Tyriar]

@wez afaik fig is based on using shell integration to extract the command args and then everything is doing using these handwritten specs. My preferred approach would be to use the shell's native asutocomplete providers (TabExpansion2/etc. for pwsh, zsh-autocomplete, bash complete, etc.), instead of re-inventing the wheel and having a parallel set of completions that comes with versioning problems.

In VSCode this is currently behind an experimental flag "terminal.integrated.shellIntegration.suggestEnabled": true, but it's kind of broken atm had some critical work that took my attention away for now.

[j4james]

Unless you have a boat load of shells that are lining up to use this new functionality, I'm of the opinion that we're better off just hacking something on top of the FTCS sequences (which I'm assuming is what fig is doing). That doesn't require any buy-in from the shells, other than the ability to modify the prompt, which even cmd.exe can handle.

Otherwise we're guaranteed to get people complaining about the functionality not working in their favorite shell. And we can't blame the shell if there other terminals or tools that can achieve more or less the same thing without any special shell cooperation.

Although maybe there is a middle ground, where we start with an FTCS-based approach (using some minimalist autocomplete specs bundled with the terminal), but a shell can still provide additional context and custom suggestions (with your new escape sequence) if they want to enhance that experience.

[zadjii-msft]

I like half agree but also half don't. I really don't love the fig-style completion specs that are external to the actual source of the command. That just sounds like a versioning nightmare to me. Plus, we'd have to also maintain a enormous list of completion data, which would suddenly stop working as soon as you encounter a previously un-spec'd command.

That being said, you're right. I don't suspect there are a lot of other shells lining up to add this / support this. I'm pretty confident there'd never be a way to support something like this for cmd.exe.

I think the middle-ground approach you described is something I'd more like to get to in the long-term. This UI is something I'm gonna start using for a whole bunch of stuff real soon, not just shell completions. In the fullness of time, I want extensions to be able to plumb their own suggestions into it. So you could have a fig extension, and then hit your figSuggestions keybinding and that would let fig give back suggestions based on the CWD and currently typed command (as powered by FTCS commands).

But I think there's room for everyone here. I think there's also space for the shell to give definitive suggestions, if it knows them. Maybe not something built-in to zsh, but something that a plugin could invoke. I really need to go research more how completions work for other shells. Do I dare start cc'ing devs on other shells? I may need to 😬

[Tyriar]

Something like that is what I was thinking, first class support for native shell completion engines and allowing fig specs to provide fallback completions via an extension.

[Araxeus]

I really need to go research more how completions work for other shells

Nushell has custom completions: https://www.nushell.sh/book/custom_completions.html

I love how they handle custom completions, that could even enable AI powered suggestions pretty easily

I'm using their recommended external completion engine https://github.com/rsteube/carapace-bin and it's working great in windows terminal (here's a nice blog post about carapace)

Screenshot of my setup

[Welding-Torch]

James McParlane has had success implementing a kind of suggestions system, and it is running on Windows Terminal. This video released just 2 weeks ago, so I thought I'd bring it to the attention of the developers here.

[zadjii-msft]

Alright, so notes on enabling the version we shipped in 1.19 are in https://github.com/microsoft/terminal/wiki/Experimental-Shell-Completion-Menu.

I'm fully intending to replace that sequence, but that's a place to start playing with it.

[AmyJeanes]

Thank you @zadjii-msft and team it looks great!

Unfortunately I've not been able to get it working after setting up shell integration which does work, ctrl+space just appears to do nothing 😢

I've debugged it to ensure that it is actually triggering PowerShell (7.3.7) to send the completions by removing the special character markers and the setting ("experimental.enableShellCompletion": true) is definitely on at the root of the config file.

Windows Terminal Preview version is 1.19.2682.0

Not sure where the best place to ask for help with that is, in this issue, a new issue or a discussion - cheers

[zadjii-msft]

WELP We're doing a bug bash with the team right now and iT wOrKs On My MaChInE but it doesn't on @carlos-zamora's and we're trying to debug as we speak

[zadjii-msft]

omfg I typo'd the wiki.

It's experimental.enableShellCompletionMenu

[AmyJeanes]

Thanks it works now!! 😄