Use VT for COOKED_READ_DATA

[lhecker]

By rewriting COOKED_READ_DATA to use VT for its output we make it possible to pass this VT output 1:1 straight to the hosting terminal if we're running under ConPTY. This is also possible with the current console APIs it uses, but it's somewhat janky. In particular the usage of ReadConsoleOutput to backup/restore the popup contents could be considered bad faith "rules for thee, not for me", given that we're telling people to move away from those APIs.

The new implementation contains a bare bones "pager" to fit even very long prompt contents into the VT viewport. I fully expect this initial PR to not be entirely bug free, because writing a proper pager with line wrapping is a little bit complex. This PR takes some significant shortcuts by leveraging the fact that the prompt line is always left-to-right and always a series of fully filled lines followed by one potentially semi-full line. This allows us to skip using a front/back-buffer for diffing the contents between two redisplay calls.

Part of #14000

Validation Steps Performed

• ASCII input
• Chinese input (中文維基百科) ✅
• Surrogate pair input (🙂) ✅
• In cmd.exe
  • Create 2 files: "a😊b.txt" and "a😟b.txt"
  • Press tab: Autocomplete to "a😊b.txt" ✅
  • Navigate the cursor right past the "a"
  • Press tab twice: Autocomplete to "a😟b.txt" ✅
• Execute printf(" "); gets(buffer); in C (or equivalent)
  • Press Tab, A, Ctrl+V, Tab, A ✅
  • The prompt is " A^V A" ✅
  • Cursor navigation works ✅
  • Backspacing/Deleting random parts of it works ✅
  • It never deletes the initial 4 spaces ✅
• Backspace deletes preceding glyphs ✅
• Ctrl+Backspace deletes preceding words ✅
• Escape clears input ✅
• Home navigates to start ✅
• Ctrl+Home deletes text between cursor and start ✅
• End navigates to end ✅
• Ctrl+End deletes text between cursor and end ✅
• Left navigates over previous code points ✅
• Ctrl+Left navigates to previous word-starts ✅
• Right and F1 navigate over next code points ✅
  • Pressing right at the end of input copies characters from the previous command ✅
• Ctrl+Right navigates to next word-ends ✅
• Insert toggles overwrite mode ✅
• Delete deletes next code point ✅
• Up and F5 cycle through history ✅
  • Doesn't crash with no history ✅
  • Stops at first entry ✅
• Down cycles through history ✅
  • Doesn't crash with no history ✅
  • Stops at last entry ✅
• PageUp retrieves the oldest command ✅
• PageDown retrieves the newest command ✅
• F2 starts "copy to char" prompt ✅
  • Escape dismisses prompt ✅
  • Typing a character copies text from the previous command up until that character into the current buffer (acts identical to F3, but with automatic character search) ✅
• F3 copies the previous command into the current buffer, starting at the current cursor position, for as many characters as possible ✅
  • Doesn't erase trailing text if the current buffer is longer than the previous command ✅
  • Puts the cursor at the end of the copied text ✅
• F4 starts "copy from char" prompt ✅
  • Escape dismisses prompt ✅
  • Erases text between the current cursor position and the first instance of a given char (but not including it) ✅
• F6 inserts Ctrl+Z ✅
• F7 without modifiers starts "command list" prompt ✅
  • Escape dismisses prompt ✅
  • Entries wider than the window width are truncated ✅
  • Height expands up to 20 rows with longer histories ✅
  • F9 starts "command number" prompt ✅
  • Left/Right replace the buffer with the given command ✅
    • And put cursor at the end of the buffer ✅
  • Up/Down navigate selection through history ✅
    • Stops at start/end with <10 entries ✅
    • Stops at start/end with >20 entries ✅
    • Scrolls through the entries if there are too many ✅
  • Shift+Up/Down moves history items around ✅
  • Home navigates to first entry ✅
  • End navigates to last entry ✅
  • PageUp navigates by $height items at a time or to first ✅
  • PageDown navigates by $height items at a time or to last ✅
• Alt+F7 clears command history ✅
• F8 cycles through commands that start with the same text as the current buffer up until the current cursor position ✅
  • Doesn't crash with no history ✅
• F9 starts "command number" prompt ✅
  • Escape dismisses prompt ✅
  • Ignores non-ASCII-decimal characters ✅
  • Allows entering between 1 and 5 digits ✅
  • Pressing Enter fetches the given command from the history ✅
• Alt+F10 clears doskey aliases ✅
• In cmd.exe, with an empty prompt in an empty directory: Pressing tab produces an audible bing and prints no text ✅
• When Narrator is enabled, in cmd.exe:
  • Typing individual characters announces only exactly each character that is being typed ✅
  • Backspacing at the end of a prompt announces only exactly each deleted character ✅

[github-actions[bot]]

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log, or :memo: job summary for details.

Unrecognized words (1)

colorbrewer

Previously acknowledged words that are now absent

codenav CRLFs Redir wcsicmp 🫥

To accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands

... in a clone of the [email protected]:microsoft/terminal.git repository on the dev/lhecker/14000-vt-cooked-read branch (:information_source: how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/v0.0.22/apply.pl' |
perl - 'https://github.com/microsoft/terminal/actions/runs/9574382579/attempts/1'

Available :books: dictionaries could cover words (expected and unrecognized) not in the :blue_book: dictionary

This includes both expected items (2213) from .github/actions/spelling/expect/04cdb9b77d6827c0202f51acd4205b017015bfff.txt .github/actions/spelling/expect/alphabet.txt .github/actions/spelling/expect/expect.txt .github/actions/spelling/expect/web.txt and unrecognized words (1)

Dictionary	Entries	Covers	Uniquely
cspell:cpp/src/lang-jargon.txt	11	1	1
cspell:swift/src/swift.txt	53	1	1
cspell:gaming-terms/dict/gaming-terms.txt	59	1	1
cspell:monkeyc/src/monkeyc_keywords.txt	123	1	1
cspell:cryptocurrencies/cryptocurrencies.txt	125	1	1

Consider adding them (in .github/workflows/spelling2.yml) for uses: check-spelling/[email protected] in its with:

      with:
        extra_dictionaries:
          cspell:cpp/src/lang-jargon.txt
          cspell:swift/src/swift.txt
          cspell:gaming-terms/dict/gaming-terms.txt
          cspell:monkeyc/src/monkeyc_keywords.txt
          cspell:cryptocurrencies/cryptocurrencies.txt

To stop checking additional dictionaries, add (in .github/workflows/spelling2.yml) for uses: check-spelling/[email protected] in its with:

check_extra_dictionaries: ''

Errors (2)

See the :open_file_folder: files view, the :scroll:action log, or :memo: job summary for details.

:x: Errors	Count
:x: check-file-path	1
:x: ignored-expect-variant	3

See :x: Event descriptions for more information.

:pencil2: Contributor please read this

By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.

If the listed items are:

• ... misspelled, then please correct them instead of using the command.
• ... names, please add them to .github/actions/spelling/allow/names.txt.
• ... APIs, you can add them to a file in .github/actions/spelling/allow/.
• ... just things you're using, please add them to an appropriate file in .github/actions/spelling/expect/.
• ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in .github/actions/spelling/patterns/.

See the README.md in each directory for more information.

:microscope: You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. :wink:

If the flagged items are :exploding_head: false positives

If items relate to a ...

• binary file (or some other file you wouldn't want to check at all).

  Please add a file path to the excludes.txt file matching the containing file.

  File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

  ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

• well-formed pattern.

  If you can write a pattern that would match it, try adding it to the patterns.txt file.

  Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

  Note that patterns can't match multiline strings.

[DHowett]

qq: x offset?

[DHowett]

One question! The old popup style allows you to see the F7 command list behind the F9 prompt, when you are using both:

I wonder if this helps you enter a number - you can see which command it lines up with.

The new one does not allow for that:

[DHowett]

Ah, another horror from beyond the known universe

this arrow character is in the ambiguous width section, which is quite fun for users of the font NSimSun

[lhecker]

Should be much better now!

[DHowett]

Tab completing something that doesn't exist if you have a control character in your input line does a funny thing too.

[DHowett]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[lhecker]

I find it remarkable that the ARM64 tests keep failing in the exact tests that I had to patch for #17510, but in this PR, I didn't change any of that code yet, so why is it failing? And why does it work in the x64 tests?? I think it's just a fluke. In any case, it'll get fixed by #17510.