Add keyboard protocol support (the kitty one)

[WSLUser]

trafficstars

Description of the new feature/enhancement

Provide a keyboard protocol for reporting raw keyboard input, preferably the Kitty keyboard protocol but at the very least XTMODKey, which Xterm supports.

Also see https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/1

Proposed technical implementation details (optional)

[zadjii-msft]

Is there a particular gap that win32-input-mode (#4999) doesn't cover that you're hoping to address?

[WSLUser]

These are based on VT CSI which I think win32-input-mode wouldn't provide (as that's specifically for win32, not VT applications). It's technically the same, but different audience. Some popular editors do appear to make use of them already, so providing the support on the VT side would be nice.

Also most notably, notcurses has support for the kitty keyboard protocol and minimum support for XTMODKey (it'll likely be re-iterated as time permits).

Also see https://github.com/microsoft/terminal/issues/4999#issuecomment-614258718, which is basically this particular request.

[WSLUser]

https://github.com/microsoft/terminal/pull/11264 also would reveal the need for this, if we want to do independent things from conhost (which you seemed to have wanted from the your reply to the comment linked above).

[WSLUser]

I guess the best proposol would be to re-work the input mode so that the win32 input would convert to one of the 2 proposed keyboard protocols. Easier would be to simply have 2 different ones, but that's asking for bad code health and inefficiencies, when much of what's already there can likely be re-used.

[zadjii-msft]

Sure okay, so this is on the client read input side, not the terminal write input side. Just "add support for {whatever} VT Input mode". That's fine, just a little different encoding from the other VT input types we have. Thanks for helping clarify!

[WSLUser]

Yeah I'm thinking https://github.com/microsoft/terminal/pull/11384 would probably help make this easier to implement as well. Perhaps @j4james would be interested in adding this support.

[j4james]

I'm only interested in implementing the standard ones, like DECPCTERM and DECEKBD.

Also, I think the XTerm protocol is closely related to #8719 (possibly intended to be the same thing, but I was under the impression that they weren't strictly compatible).

[WSLUser]

Yes Iterm is slightly incompatible with Xterm's implementation, which is why I'd rather we opted for Kitty's instead as the most robust one existing and upon which the terminal working group was discussing as a base to which certain changes were encouraged by some terminal authors. Never got to resolution though and though it's the first one in the gitlab, it's also the longest standing one still awaiting a standard. I do know there are a number of TUI applications that make use of Kitty's keyboard protocol as issues were raised about it in the past, so we'd be getting the support of those applications automatically. We could also adopt both but that makes little sense as they aim to do the same thing, just one does it better than the other.

[j4james]

I'd rather we opted for Kitty's instead as the most robust one existing and upon which the terminal working group was discussing as a base

If you read the whole terminal-wg thread, most people seem to be opposed to the Kitty protocol and were arguing for some version of the CSI u protocol instead. Do you know of any terminals other than Kitty that implement it?

[zadjii-msft]

Yea frankly, libtickit has the superior protocol. I'm only leaving this open as a "if someone really, REALLY wanted to implement this for us, we'd accept it". It's helpful for folks to know what still needs to be done, even if we're not planning on getting to it ourselves.

[unxed]

The problem with the CSI u (libtickit) protocol is that there is no way to specify alternate layout key[s]. This is useful for keyboard layouts such as Cyrillic where you want the shortcut ctrl+c to work when pressing the ctrl+С on the keyboard. Kitty's "Report alternate keys" mode fixes it.

[kirawi]

So far, these terminal emulators have implemented the protocol:

• Kitty
• Foot
• Alacritty
• Rio (referenced Alacritty's implementation)
• Wezterm

I think that with these recent additions, not opting for the kitty enhanced keyboard protocol has become a more contentious path than when this issue was opened.

[j4james]

In case I'm not the only one that wasn't aware of this, it's worth noting that the current kitty keyboard protocol which everyone is implementing is not the same as the one that was originally proposed in terminal-wg. I think the switch actually happened years ago, but I only found out about the new version fairly recently.

I haven't looked at it in much detail, but from my initial impression I'd now be more inclined to pick kitty over libticket/xtmodkeys if we were going to implement one of them.

That said, this is still not something I'm personally interested in at this point in time.

[unxed]

Despite the fact that the kitty keyboard protocol seems overly complicated and difficult to implement, writing its implementation turned out to be quite easy.

Here is the implementation code I wrote for far2l, I am releasing this code snippet under the Public Domain license so that anyone can use it.

This is not a complete implementation, but it can be easily extended to become a full one. However, this implementation is sufficient for all the applications I have tested (far2l file manager, turbo text editor, Free Pascal’s Free Vision sample app with kitty keyboard protocol support patch from their GitLab). At the very least, this implementation can be used as a reference.

This code generates kitty protocol ESC sequences from Windows-compatible Key Event Record structure and can be easily adapted for use in the Microsoft terminal.

[determin1st]

can you just design a better protocol? why so copycatish

[unxed]

As for me, win32 input mode is fine, and we already support it in far2l. As for alternate layout feature, virtual key code field can be used for that.

Btw, it is important to understand that a protocol without applications that use it makes no sense, and the kitty's protocol already has a certain set of applications that work with it.

[determin1st]

FAR is a great program, but is not a library layer that gives a terminal software an abstraction layer. i personally see no point in bending to poor designs.

[unxed]

Libraries implementing kitty keyboard protocol:

The notcurses library The crossterm library The textual library

[determin1st]

oh, whata enshittification of the nix systems. nope, i wont do that.

windows has a ReadConsoleInput API at least i dont see whats underneath those

i spoke to kittys author, he doesnt understand the benefit of 8-bit parser, thats enough to qualify his designs you blindly adopted

[zadjii-msft]

Thanks for the reference implementation! If there was an enterprising individual out there that was really interested in contributing this, I'd take a look at

https://github.com/microsoft/terminal/blob/main/src/terminal/input/terminalInput.hpp

TerminalInput is the class that's responsible for converting all INPUT_RECORDs into VT sequences. You'd probably also need to handle the CSI > 1 u somewhere in https://github.com/microsoft/terminal/blob/main/src/terminal/parser/OutputStateMachineEngine.cpp and https://github.com/microsoft/terminal/blob/main/src/terminal/adapter/adaptDispatch.cpp

[unxed]

Regarding the disadvandages of the kitty protocol, I see only two at the moment.

The specification seems overly complex. The real issue here lies in how the specification is written, rather than the protocol itself. I'll explain its essence to you in just a few words.

Each key event message in this protocol contains 7 parameters (first 5 are of type int, next goes one or more ints separated by colons and the last parameter is of type char), of which only the first and last ones are mandatory. Messages have the following structure:

ESC [ unicode-key-code : shifted-key : base-layout-key ; modifiers : event-type ; text-as-codepoints suffix

Example:

ESC [122;129:3;122u

The presence of certain fields is determined by, on one hand, their non-zero value, and on the other, by the flags set when activating the protocol.

• unicode-key-code is the character corresponding to the pressed key, in lowercase, or special code point representing non-text key, according to the table in the specification.
• shifted-key is the same character considering the shift and caps lock keys.
• base-layout-key is the character that the same key would print when the English layout is set.
• modifiers is a bitfield containing information about the pressed modifiers, such as alt or ctrl.
• event-type is the type of event: press, release, or repeat (in case the key is held down).
• text-as-codepoints contains one or more Unicode code points corresponding to the pressed key, in the form that the text would be sent if the kitty protocol was not enabled.
• suffix is added for compatibility reasons and can differ from key to key, table is provided in spec

That's it! The protocol is actually so simple that you already understand how it works.

The specification for the modifiers field does not distinguish between right and left modifier keys. Fortunately, this can be easily worked around on the application side by tracking the press and release events of the right modifier keys. Also the protocol can easily be extended adding support for handling right modifiers keys better while still maintaining full backward compatibility.

Considering all this, as well as the highest number of implementations in applications, I believe that this protocol has the best chance of becoming the de facto standard for full keyboard input support in terminals.

[determin1st]

nope, that sux terribly

first of all protocol starts with query that allows terminal application DETERMINE if it supports the protocol and maybe which version of the protocol is shall use.

those sequences are of the group of DECRQM (requested private modes). https://www.invisible-island.net/xterm/ctlseqs/ctlseqs.html (here search DECRQM)

it may look like

<CSI>?1009$p, where <CSI> is <ESC>[

so terminal receives that and replies with DECRPM, that may look like

<CSI>?1009;1y$

where 0 - not recognized 1 - set 2 - reset 3 - permanently set 4 - permanently reset

1 means okay, protocol 1009 is supported. no reply or 0 means not supported.

if you dont understand why terminal application needs that before starting protocol, well, continue your little enshittification procedures, otherwise submit and we continue discussions. why this? because it is in the docs already, that invisible land and also DEC manuals have it. or if you think it is not worthy, proppose worthy.

[zadjii-msft]

Yo bud, take a sec to refresh your reading of the code of conduct. Just because you disagree with a particular protocol's design choices doesn't make it enshittification. Enshittification would be "everyone's got to subscribe to be able to use The Good Features otherwise you're stuck with Less Good Stuff". Enshittification is not just "this app added support for a feature I disagree with".

There's literally no harm in someone adding support for this protocol. A broader set of compatibility? Great! Adding support for this protocol does not remove support for any other protocol. They can all exist in parallel. I'm not gonna do it myself, but I'm not gonna stop the community from helping build the best possible terminal. Obviously, adding support for this input protocol would also involve support for the DECRPM query. https://github.com/microsoft/terminal/blob/760daa642ed26935d1621548532ff9e03fe474a5/src/terminal/adapter/adaptDispatch.cpp#L1957-L1964

[determin1st]

i think i fully comply with the codes, yo, because ima gay negro in a wheelchair

yes, i just disagree as you said. enshittification simply means degradation from the previous state. what was it? it was ReadConsoleInputA ive mentioned. youre microsoft-dude, but you didnt see it is worse? cant you say it worse because of codes? see, me is very convenient, i said it for you.

yes, i want design better protocol but i see only ambassadors imposing theirs, they dont want to design better. they dont want to communicate.

[denelon]

@determin1st I ran across this issue and the concern around the code of conduct.

Let's be sure we are being respectful of differing opinions, viewpoints, choices and experiences. Constructive, feedback is the best way to partner together. Our repositories are safe spots for sharing ideas.

Examples of unacceptable behavior include:

• Trolling, insulting or derogatory comments, and personal or political attacks

[denelon]

The sections of the discussion marked as "off-topic" appear to be focused on an implementation choice to add support for an existing keyboard protocol (the kitty one) as opposed to designing and implementing a new protocol. This does not appear to be a mutually exclusive decision. Many protocols could be supported. Adding support for another existing protocol does not preclude adding support for a new protocol in the future.

This GitHub issue is regarding adding support for an existing protocol (the kitty one). There is nothing wrong with having a differing opinion and proposing an alternative existing protocol or suggesting the design and implementation for a new protocol. That discussion appears to be "off-topic" in terms of how it affects the implementation of the existing protocol (the kitty one).

One way to keep this GitHub Issue on topic is to add a comment with a new linked GitHub Discussion to take the "off-topic" content to a separate GitHub Issue or GitHub Discussion. I don't know that the Microsoft Terminal project is the best place to discuss the design for a new protocol, but it's a great place to start that discussion since this GitHub Issue spawned the thread. I've created a discussion to discuss Designing a new keyboard protocol

If there is interest in designing a new protocol to be added to Windows Terminal a new project could be created and linked in that discussion. Since it appears that the maintainers of Windows Terminal are willing to allow the addition of the existing protocol. I've created a new discussion so others who might be interested in working on a new keyboard protocol can join and decide how to proceed.

[kirawi]

From my understanding, Windows already has its own keyboard protocol? This is mentioned in the beginning of the issue. https://github.com/microsoft/terminal/blob/main/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md

I think Wezterm has support for it too.

[determin1st]

okay, lets read what microsoft-dude invented

From my understanding, Windows already has its own keyboard protocol? This is mentioned in the beginning of the issue. https://github.com/microsoft/terminal/blob/main/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md

i just stop reading after this

Keys that we definitely need to support, that don't have unique VT sequences: Ctrl+Space (https://github.com/microsoft/terminal/issues/879, https://github.com/microsoft/terminal/issues/2865) Shift+Enter (https://github.com/microsoft/terminal/issues/530) Ctrl+Break (https://github.com/microsoft/terminal/issues/1119) Ctrl+Alt+? (https://github.com/microsoft/terminal/issues/3079) Ctrl, Alt, Shift, (without another keydown/up) (https://github.com/microsoft/terminal/issues/3608, https://github.com/microsoft/terminal/issues/4334, https://github.com/microsoft/terminal/issues/4446)

cmd.exe :]]

what you "inventing" here is worse than ancient ReadConsoleInput i dont understand the need for topic creation, i can discuss protocols anywhere but i see no other users. wheres kitty-dude inventor? wheres those nocurses brightercurses authors? im not gonna talk with managers

btw i told him its not INPUT_RECORD, which is union type, no reaction

[o-sdn-o]

@determin1st The protocol https://github.com/microsoft/terminal/blob/main/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md was created to forward the ReadConsoleInput functionality 1 to 1 in the VT environment. This protocol is a wrapper for ReadConsoleInput and supports all the keyboard events that ReadConsoleInput does. No more and no less.

I agree with you that more is needed.

There are a number of keyboard events within the system that we need to track that are not covered by ReadConsoleInput, nor any existing Win32 Console API routine or VT sequences, nor even any existing keyboard protocols, including kitty. So we need to come up with a new protocol.

Support for the kitty protocol is necessary because we need to run applications that rely on this protocol here and now.

[gdamore]

Kitty Protocol mode is gaining broader acceptance.

Right now I have a problem where I have an unfortunate choice for my users:

1. Enable VT input mode, so that I can use bracketed paste mode, and lose the richer support for handling Control characters, etc. that the input events from Windows gives me (e.g. accept that I cannot discriminate between Control-I and TAB, for one example, and won't receive key codes for a lot of other keys)
2. Use Input Events, but lose any hope of being able to get bracketed paste.

If we had Kitty mode, then I could use option 1. Alternatively, if we could close the gaps so that win32-input-mode provided a strict superset of functionality provided by VtInput mode (and right now I'm specifically talking about bracketed paste), then I'd be happier indeed!