Switch terminal backend from Crossterm to Termina

[the-mikedavis]

Termina is a new terminal manipulation crate I've written that has a similar interface as Crossterm. The interface exposes lower-level details than Crossterm's, exposing escape sequences and pushing all of that handling to the application.

Termina has a minimal Windows integration that works because of ConPTY - we tell terminals on Windows to "speak VT" like a *NIX PTY would. This drops a lot of Windows-specific code for the old Console API and allows us to support VT features consistently on all platforms. In particular OSC52 bracketed paste should work on Windows now. The downside is that we will require Windows 10 like WezTerm: version 10.0.17763 or later. This is a fairly old version though, published ~Fall 2018. The Kitty keyboard protocol should work on WezTerm (when enabled in config) now, ~though it seems a bit buggy in my testing - needs some debugging~.

Termina also has an interface for polling and reading for arbitrary events and escape sequences. (This exists in Crossterm too but is private.) This allows us to detect VT extensions like synchronized rendering and true color support while setting up the terminal.

Rendering should be smoother on this branch since synchronized output sequences are enabled when we detect support for them (fixes #731). Termina also has better "compression" for SGR sequences (things like foreground/background/underline colors, bold, italic, etc.) similar to https://github.com/crossterm-rs/crossterm/pull/879 but covering all SGR attributes in addition to foreground and background - this should give the terminal less work to do when it parses the escapes we emit during rendering.

This needs some testing. I did some limited manual testing on Kitty and Ghostty on macOS, Kitty, Foot and WezTerm on Linux and cmd.exe, PowerShell and WezTerm on Windows.

Also fixes #12232 (the same way as #13223 did)

[ghost]

Wow that is a noticeable improvement 🚀

I have some issues with space+y copying though.

I'm using Ghostty (1.1.3) on MacOS (arm) and helix is running ~~in a Zellij (0.41.1) session~~ (reproducible without the zellij layer) over SSH on a NixOS (arm) server.

Copying with the termcode (OSC52) provider randomly stops working (it is easily reproducible, when copying multiple times) and I sometimes get ~~random text output (looks like base64 encoded, but I get gibberish when I try to decode)~~ the base64 encoded clipboard content in the buffer after the current selection which disappears when I select the lines.

Also checked the main branch again and it still works as expected.

Let me know if you need further information.

[RoloEdits]

I see you linked https://github.com/crossterm-rs/crossterm/pull/879, would the performance gains seen there benefit Windows this time now that the terminal is "spoken" to the same way on all platforms?

[the-mikedavis]

Copying with the termcode (OSC52) provider randomly stops working

I was able to reproduce this with the same setup. I changed the OSC52 code a bit to do some explicit locking and flushing and I haven't been able to reproduce it so far. That part is a bit hacky - we basically print!() the command instead of executing it on the terminal buffer. We would need a bigger refactor to find a way to fix this properly.

I also noticed that true-color detection was broken for me in that workflow (i.e. SSH with Ghostty. Probably I would need to install the Ghostty terminfo on the NixOS side). I've updated the true-color detection to use a query following https://github.com/termstandard/colors#querying-the-terminal so it works out of the box in this case.

would the performance gains seen there benefit Windows this time...?

Crossterm does actually "speak VT," meaning that it emits escape sequences - like those used to set foreground/background colors - when it detects that ConPTY is available. The issue with its Windows handling is that it doesn't "hear VT": it enables ENABLE_VIRTUAL_TERMINAL_PROCESSING on the output handle (CONOUT$ or stdout) but not ENABLE_VIRTUAL_TERMINAL_INPUT on the input handle (CONIN$ or stdin) (Microsoft docs). So Crossterm was already using that optimization in https://github.com/crossterm-rs/crossterm/pull/879 on Windows as well. It lacked the ability to read VT inputs from ConPTY though, so we wouldn't ever get a response to request/response type escape codes like DECRQSS/DECRPSS (what I was mentioning in https://github.com/helix-editor/helix/pull/13224#issuecomment-2781513200 and what makes the true color thing above possible) or detecting support for the Kitty keyboard protocol or synchronized outputs.

[RoloEdits]

Got some time to play around with this and I see some odd characters on the status bar (where I expect different ones) and when I press tab I see this:

Windows 11, WezTerm, Nushell

This was with:

config.enable_kitty_keyboard = false -- (also tried true and same thing)
config.allow_win32_input_mode = true-- (also tried false with kitty true and same thing)

2025-04-09T19:06:28.167 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 3.1107ms: Capabilities { kitty_keyboard: None, synchronized_output: true, true_color: false, extended_underlines: false }

The keys seem to work now, with the protocol enabled, where previously it didnt, but there is just this character glitch.

edit: actually when trying more keys, the arrow keys dont work. hjkl do still.

[chtenb]

On windows 11 it hangs during startup on wezterm, but launches correctly on windows terminal. The logs don't show anything.

[cotneit]

Not sure how I can best help with debugging these issues but here are some of my findings.

Windows Console Host

Tried conhost.exe (Windows allows switching between conhost and Windows Terminal in Settings > System > For Developers) with nushell, cmd.exe, powershell.exe (powershell that comes with Windows), pwsh.exe (new powershell) and they all hang like this:

Windows Terminal and Wezterm

Experiencing issues in Windows Terminal and WezTerm (nushell, powershell, cmd - looks like same issues in all shells)

• Trying to open command palette, pressing Shift to input ? causes keybind popup thingy to close, so ? gets invoked instead of <space>+?
• Can't quit with :q!, looks like a space gets inserted when I do Shift + 1 to insert ! resulting it :q !
• Mouse buttons and scroll wheel do not work
• Arrow keys don't work

Windows Terminal: Version: 1.22.10731.0

2025-04-10T17:12:10.169 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 114.7µs: Capabilities { kitty_keyboard: None, synchronized_output: false, true_color: true, extended_underlines: true }

WezTerm: Version: wezterm 20250131-201646-b28bbbc4

2025-04-10T17:43:24.584 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 102.6µs: Capabilities { kitty_keyboard: None, synchronized_output: false, true_color: true, extended_underlines: true }

[the-mikedavis]

The unicode stuff should be fixed in the latest push - I was accidentally resetting the input/output "code pages" (encodings) after querying for features. That should cover the weird glyphs for non-ascii characters and input problems like :q!. I'll spend some time on Windows today looking at the rest.

[cotneit]

~~Awesome, everything I mentioned above aside from conhost is fixed now!~~

Jumped the gun a little, fixed in Windows Terminal but these are still an issue in WezTerm it seems:

• Trying to open command palette, pressing Shift to input ? causes keybind popup thingy to close, so ? gets invoked instead of +?
• Can't quit with :q!, looks like a space gets inserted when I do Shift + 1 to insert ! resulting it :q !

Basically every time CTRL/SHIFT/ALT/WIN keys are pressed a space(?) is inserted, at least in input mode and prompt

[the-mikedavis]

What does your WezTerm config look like?

[cotneit]

Updated WezTerm to latest nightly and the issue disappeared

Also tried stable wezterm 20240203-110809-5046fc22 and it behaves same as conhost - hangs after hx is executed

So to sum it up:

• wezterm 20250320-072107-a8735851 (latest nightly) - works
• wezterm 20240203-110809-5046fc22 (stable) - hangs
• wezterm 20250131-201646-b28bbbc4 - bug described above

My config is pretty basic:

return {
    color_scheme = 'tokyonight',
    default_prog = { 'pwsh.exe', '-NoLogo' },
}

[cotneit]

Tried conhost and stable WezTerm after last commit, they no longer hang but noticed the following issues in both:

• No colors
• Same issue with CTRL/SHIFT/ALT/WIN/CAPSLOCK inserting spaces as described above

UPD: Fixed

[RoloEdits]

This is all with wezterm:

The text issue was fixed, but now no capabilities seem to have been detected

2025-04-10T12:39:55.065 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 3.7429ms: Capabilities { kitty_keyboard: None, synchronized_output: false, true_color: false, extended_underlines: false }

With allow_win32_input_mode set to false, ESC no longer works, like it did previously (with the text glitching).

For me, capslock works as expected. ctrl and alt also seem to be fine. Colors are fine.

[RoloEdits]

latest commit looks more promising with the capability check, though I wonder why true color and underlines are false?

2025-04-10T13:32:29.740 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 2.708ms: Capabilities { kitty_keyboard: Some, synchronized_output: true, true_color: false, extended_underlines: false }
2025-04-10T13:32:29.753 helix_tui::backend::termina [DEBUG] The terminal fully supports the requested keyboard enhancement flags

[the-mikedavis]

For better compatibility I switched to the foreground/background sequences suggested in the microsoft docs: https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#extended-colors. (As I understand it it's not spec-wise correct but is an idiom used in the wild more often than the correct syntax.) That should fix the colors on conhost and stable WezTerm (which is quite old, probably it bundles an old conhost). It's the same format as crossterm uses so it should be the same compatibility-wise.

That last push should fix the detection: out of the box WezTerm should be true for synchronized output and false or None for everything else (or Some if you have the kitty keyboard flag enabled). It does support true color and underlines but it doesn't currently respond to a request that would tell us if it did (that would be https://github.com/wezterm/wezterm/pull/6856). Probably the log message should be clear that false actually means "no response to the query" rather than a definitive lack of support.

[RoloEdits]

I did notice some weird behavior with escaping and scrolling as well as just in general with escape, in that pressing it seems to cause some kind of delay. Or like the press doesnt actually "happen" until I give another input? Kind of hard to show this, but you can see me scrolling when you see the rapid change in key inputs at the bottom right. A menu also pops up when all I am doing is scrolling. Also when the prompt is open I am spamming escape but nothing happens until I press, in this instance, an arrow key:

which is quite old, probably it bundles an old conhost

That is correct, nightly now has one from this year, just a few weeks old I believe.

[the-mikedavis]

Is that with the Kitty keyboard protocol enabled? I've seen that behavior with WezTerm as well when it is enabled - termina doesn't seem to get anything from the input handle until another button is pressed - but I haven't debugged why yet. I don't see the same in Alacritty notably and I believe it has at least some support for the Kitty keyboard protocol.

[RoloEdits]

Is that with the Kitty keyboard protocol enabled?

That is correct, the behavior shown is with these set:

config.enable_kitty_keyboard = true
config.allow_win32_input_mode = true

[the-mikedavis]

It looks like this is a bug in the Kitty keyboard encoding function in WezTerm. I sent a patch https://github.com/wezterm/wezterm/pull/6872. I'm not sure why the old encoding causes the key to 'queue up' and wait for a non-escape input but using the proper representation seems to fix it.

[RoloEdits]

Ok, added those wezterm patches and now running them.

2025-04-11T22:34:56.878 helix_tui::backend::termina [DEBUG] Detected terminal capabilities in 2.5524ms: Capabilities { kitty_keyboard: Some, synchronized_output: true, true_color: true, extended_underlines: true }
2025-04-11T22:34:56.889 helix_tui::backend::termina [DEBUG] The terminal fully supports the requested keyboard enhancement flags

All capabilities seem to be detected. The keys and mouse seem to work correctly. I will run this more and get back if anything is up.

Thanks for getting those upstream fixes in, the time and effort you spent is greatly appreciated.

[RoloEdits]

Took me a minute to figure out what I was feeling was off, kind of a subtle thing when you haven't found it: the undercurl seems to be resetting or being ignored till its updated.

"diagnostic.error" = { underline = { color = "red", style = "curl" } }
"diagnostic.warning" = { underline = { color = "yellow", style = "curl" } }
"diagnostic.info" = { underline = { color = "sky", style = "curl" } }
"diagnostic.hint" = { underline = { color = "teal", style = "curl" } }
"diagnostic.unnecessary" = { modifiers = ["dim"] }
"diagnostic.deprecated" = { modifiers = ["crossed_out"] }

[chtenb]

On windows 11 it hangs during startup on wezterm, but launches correctly on windows terminal. The logs don't show anything.

This seems to be resolved on the current tip.

[the-mikedavis]

That underline behavior may be a bug with WezTerm, I haven't been able to reproduce on Kitty, Alacritty or Foot yet. I'll take a look at how WezTerm is parsing those underline commands

[archseer]

I remember we were dealing with a wezterm bug like that before, I think it depended on the order of the style data

[archseer]

Previously seen here: https://github.com/helix-editor/helix/pull/4061#issuecomment-1275407947

[the-mikedavis]

I believe that was a different issue with the crossterm backend at the time since I could reproduce it then on different terminals. I added a commit to work around the problem. It seems like WezTerm doesn't like having the underline color included in the big SGR update that updates all attributes that changed between two cells. We can work around it for now by emitting a separate SGR update whenever the underline color changes.

[j4james]

It seems like WezTerm doesn't like having the underline color included in the big SGR update that updates all attributes that changed between two cells.

This is likely because of a parameter limit. WezTerm has a limit of 16 (at least last I tested), and attributes with subparameters count as multiple parameters. For example an RGB color sequence like \e[38;2;255;128;64m counts as 5 parameters, even though only one attribute is being changed.

16 is actually a fairly common limit, and it's not even the lowest! I believe MLTerm has a limit of 10. That's just enough for two RGB colors, but if you're trying to change other attributes at the same time they're liable to get dropped or corrupted. So you need to be very careful how much you change in one sequence.

[the-mikedavis]

Aha, that makes sense, thanks! I'll update this escape to "chunk" the updates so they stay under that conservative limit. If we find lower limits we can tune that further.

[the-mikedavis]

Looks like WezTerm just bumped its CSI param limit from 32 to 256: https://github.com/wezterm/wezterm/pull/6194. It's still probably better to keep the number of parameters low for wider compatibility though

[cotneit]

I accidentally ran hx $nu.default-config-dir | path join 'zoxide.nu' instead of hx ($nu.default-config-dir | path join 'zoxide.nu')

And something broke (I'm not sure how this is supposed to work, but master seems to handle it)

This is with nushell 0.103.0 in Windows Terminal

master:

termina: (Helix is running but output is nonsense, the only way to resolve this is to close the terminal tab)

Simplified example: hx | echo master:

termina: (Helix is running but output is nonsense, the only way to resolve this is to close the terminal tab)

[the-mikedavis]

That should be fixed now after https://github.com/helix-editor/termina/commit/bd391b13fa19e22c9d09a782928491029a202bf4. Also added the "chunking" for SGR attributes