terminal
terminal copied to clipboard
microsoft
Add support for VT102 Insertion-Replace Mode (IRM)
Insertion-Replacement Mode (IRM)
ESC [ 4 h
033 133 064 150
Set selects insert mode and turns INSERT on. New display characters move old display characters to the right. Characters moved past the right margin are lost.
ESC [ 4 l
033 133 064 154
Reset selects replace mode and turns INSERT off. New display characters replace old display characters at cursor position. The old character is erased.
original issue body follows.
The following is from recording command line editing in an ssh session to a freebsd environment (via FreeBSD's script). The displayed text is odd in Windows Terminal. First I show a textual form that shows the escape sequences and such in a readable form (I eliminated bell-ring characters from the file and other lines of text):
more ~/odd_commnd_line_editing_display.txt ls ~/sys_typescriptsESC[1ESC[4h-ESC[4lESC[4hlESC[4lESC[4hTESC[4lESC[4hdESC[4lESC[4htESC[4lESC[4h ESC[4
A normal display in other environments (for example macOS terminal doing an ssh into FreeBSD) looks like:
cat ~/odd_commnd_line_editing_display.txt ls -lTdt ~/sys_typescripts/
(The above shows the actual ls command that was executed.)
But under Windows Terminal (Preview) it shows:
cat ~/odd_commnd_line_editing_display.txt ls -lTdt ~ypescripts/
This is also how it ends up looking during the command line editing, not showing the actual text of the command to be executed: missing some text.
The sequence is from deciding to add "-lTdt" after having typed the path. (It was a "set -o vi" type of command line editing context.)
I was using TERM being xterm-256color on FreeBSD .
I suspect this is because the Windows terminal doesn't yet support Insertion-Replacement Mode (IRM).
Interesting! Didn't know IRM was a thing. I'm going to repurpose this as a task to implement IRM.
Notes:
Insertion-Replacement Mode (IRM)
ESC [ 4 h
033 133 064 150
Set selects insert mode and turns INSERT on. New display characters move old display characters to the right. Characters moved past the right margin are lost.
ESC [ 4 l
033 133 064 154
Reset selects replace mode and turns INSERT off. New display characters replace old display characters at cursor position. The old character is erased.
Yet another symptom of the same bug:
Resize the window to 63 columns. Start mc (Midnight Commander) on Ubuntu, compiled against the S-Lang screen drawing library (as it's done in Ubuntu).
Expected: The bottom row should display 9 buttons, each of them being 2 characters for the function key number + 5 for the action. The rightmost should be <space>9Pu~Dn, reaching all the way to the very end.
Actual: <space>9Pu~D is printed with the expected colors, and the last column remains empty, showing the terminal's background color.
What the S-Lang library does is an ancient and obsolete hack for some terminals that don't have "pending wrap" (@j4james do you know more about them?), to print in the lower right corner without the cursor automatically wrapping to the next line and shifting everything up. It prints the intended last letter (n in this case) in the penultimate (62nd) column, and then using IRM, prints the penultimate letter (D) again in the penultimate (62nd) column, expecting it to push the previously printed n to the 63rd.
Even at bigger window sizes, such as the normal 80 columns, mc misses to paint the bottom right corner using some of its skins, e.g. F9 -> Options -> Appearance -> sand256 is one of these. For some mysterious reason, S-Lang decides to use different drawing method depending on some property of the skin. In this case, it repeats the aforementioned trick with space characters, whereas with most other themes, it just emits the escape sequence that wipes to the end.
This is a tiny inconsistency (almost-bug) somewhere in S-Lang, my wild guess is that it could be related to non-color attributes (in this case italic) being used there, but I don't think it's worth investigating S-Lang's behavior any further. Also, for the record: I couldn't reproduce the issue with mc compiled against ncurses, apparently it knows about "pending wrap" and knows that this hack is not necessary.
What the S-Lang library does is an ancient and obsolete hack for some terminals that don't have "pending wrap" (@j4james do you know more about them?)
I know DOS and the original Windows shell didn't support "pending wrap", but then they wouldn't have supported IRM mode either. However, it wouldn't surprise me if there were other third-party terminal emulators for Windows that shared that wrapping behaviour, but were still able to handle IRM, so maybe that's the sort of thing that S-Lang is targetting.
By the way, IRM is on my TODO list (assuming nobody else takes it on first). It's just that we don't yet have a handler for ANSI modes, and the existing dispatch mechanism makes that a bit of a pain to add, so that was something I was hoping to address first. There are various other issues related to parameter handling (e.g. #2101 and possibly #942), and I wanted to make sure we had something that could satisfy all of those requirements if possible, so it's probably not going to be a quick fix.
this is a a real issue. pico editor seems to use this when ran under emulated xterm. and the characters do not move to the right like they are supposed to.
still an issue as of this day.
I was actually looking into this recently, and I think it's going to require some architectural decisions before we can consider implementing anything.
I'd initially thought that the AdaptDispatch class could handle the inserting prior to writing anything, by simply scrolling the line it was about to write into by the appropriate amount. But that assumes it knows how much space the text is going to take (which it doesn't), and if the text is going to wrap onto another line, then potentially handle inserting on later lines (which may not even exist yet if vertical scrolling is required).
As things currently stand, I don't think this is possible. So either the inserting must be handled in the PrintString implementation itself (which is essentially the WriteCharsLegacy function that we don't want to touch). Or the AdaptDispatch class needs to take over the majority of the text layout, and have PrintString be a very basic operation that only writes to a single line (no inserting, wrapping, scrolling, or margin checks).
Considering the additional complications that are going to be introduced to the layout as we expand the VT functionality, I think the latter option may well be the best long term approach. But that is going to require some additional APIs, e.g. to query how much space a string of text will occupy and/or where it would be clipped.
I also have the same problem, but I have a workaround.
FreeBSD uses termcap (rather than terminfo) and the termcap entries for xterm include the 'im' and 'ei' capabilities which invoke the VT102 IRM. In the Linux terminfo database, these capabilities are not implemented.
The workaround is to create a .termcap file in your home directory with this one entry in it:
xterm-256color:mi@:IC=\E[%d@:ei@:ic=\E[@:im@:tc=xterm-new:
This will use this instead for the xterm-256color terminal (which is what Windows Terminal is), and delete the 'im' and 'ei' capabilities.
Until the IRM capabilities are implemented, this works perfectly.
It's wild how IRM works with line wrapping in xterm (351). Text is pushed off the right margin, as expected, but when the inserted region is wider than will fit it wraps and pushes text on the next line to the right.
This is a terribly specific feature. I wonder why it was introduced!
no version of xterm has ever not supported IRM. if you're not going to implement it, please identify yourself as a VT100 instead of xterm256-color. the pending-wrap thing is a side show, don't worry about it. just please make :ei=\E[4l: and :im=\E[4h: work. i've gone back to konsole as a result of this bug.
I have to use putty because of this STILL, even though putty doesn't have perfect xterm emulation either. For those of you who can't seem to reproduce this, your pico is actually nano, which has a workaround for this bug.
If you actually use the pico text editor, you can very easily see the behavior. it's the simplest test case.
The workaround is to create a .termcap file in your home directory with this one entry in it:
xterm-256color:mi@:IC=\E[%d@:ei@:ic=\E[@:im@:tc=xterm-new:Until the IRM capabilities are implemented, this works perfectly.
Thanks, your workaround fixes some weird behaviour when connecting to FreeBSD boxes. Waiting for a real fix ;-)
Marc Reymann writes:
...
Thanks, your workaround fixes some weird behaviour when connecting to
FreeBSD boxes. Waiting for a real fix ;-)
freebsd is not the only place this matters. other systems also expect that
if the offered terminal type is xterm256-color, then IRM will work. my
workaround is to use TERM=vt100, which means, my workaround is to use
konsole instead.
Great!! I was dealing with this bug since April, it was really annoying. I just apply the workaround with xterm-clear:te=\E[?1049l:ti=\E[?1049h:mi@:IC=\E[%d@:ei@:ic=\E[@:im@:tc=xterm-new: and it worked!! I hope Microsoft implements the real fix soon!!
I'm taking a crack at this.
It looks like we don't even have SM/RM, so there's a lot of plumbing to do.
@j4james the approach I've gone with -- which is of course subject to discussion and eventual change :smile: -- is to make TextBuffer do an inserting write at the cursor position. I've not made any architectural changes to Adapt or change whose job it is to do text layout. I tested gnome-terminal here & found that they don't do any re-wrapping, even if the line was emitted wrapped. Insert mode there impacts only the physical display & not the logical paragraph/wrap state.
make TextBuffer do an inserting write at the cursor position.
Yeah, but where are you doing that? In the WriteCharsLegacy implementation? And if so, that assumedly requires another OutputMode flag, or at least another WC_* flag. I just assumed we wouldn't want to be introducing more complexity there?
I was starting to think maybe now was a good time to take on #780, and incorporate the IRM functionality there. That seemed easier than worrying about how this would impact the legacy code. But maybe I'm overthinking things and you've got a simpler solution that I'm just not seeing.
It looks like we don't even have
SM/RM, so there's a lot of plumbing to do.
Yeah, that's why I wanted to get #7799 out the way first. It should at least be easer to extend the framework now than it was back then.
It would be nice if you add some spaces to see how it behaves on multi word lines.
I'm taking a crack at this.
It looks like we don't even have
SM/RM, so there's a lot of plumbing to do.
@j4james oh, right. Sorry to be inexact. Since all of the at-cursor writes coming out of WriteCharsLegacy (and any other source in conhost; see _stream.cpp:536 for the only instance I could find) go through SCREEN_INFORMATION::Write(OutputCellIterator), I put IRM determination in SCREEN_INFORMATION and kept it out of WCL. That keeps it away from any modes, control character handling, knowledge of backup/line wrapping state, etc.
Every other Write uses a coordinate target, but perhaps it would be prudent to rename this one to WriteAtCursor.
Does that seem a reasonable place?
OK that's brilliant. That's so much simpler than I was expecting. I can't see any reason why that wouldn't work. :shipit:
Please ignore results in Gnome Terminal, unless they can be reproduced under real XTerm.
IRM is designed for the case of a text editor with autowrap turned off. This saves bandwidth. The test case animated graphic is not a real world scenario, though it is what sending the raw characters to the terminal would do.
Connecting to a (Arch)Linux box has the problem. No IRM. If I isseu export TERM=vt100 vi behaves as designed. It would be great to include this as setting. Connecting with PuTTY works fine and offers a variety of settings:
Is this the root cause of cursor mode not changing?
IE: in CMD
vs
Or should this be considered a new issue?
The feature here--IRM--is unrelated to any CMD features or console applications that use the "Windows-style" cooked input feature (Python, PowerShell without PSReadline, Command Prompt).
I'm also affected by this. Is there any known way to work around it?
Yes it is setting the TERM variable to VT100
export TERM=vt100
I'm also affected by this. Is there any known way to work around it?
I am just using tmux to prevent insert mode before the last space in a command. The Issue only happens to me when editing a command from the history.
export TERM=vt100 did not change this behaviour.
Can't SSH into a lot of lab/course(from online course sites) VMs because of this.
Any workarounds for this?
export TERM=vt100 didn't work.
Numan Zaheer Ahmed wrote on 2022-12-11 15:32:
Can't SSH into a lot of lab/course(from Coursera, Udemy, etc.) VMs because of this.
Any workarounds for this?
don't use microsoft's terminal emulator. stick with securecrt, or install linux or freebsd inside "hyper-v" and run your terminal sessions inside the client VM. neither microsoft's "terminal" or "ssh" are ready for prime time.
|export TERM=vt100| did't work.
i should have been clearer. "export TERM=vt100" is what you do on the remote system after you log in. the sshd that launches your shell there is going to pass the TERM variable it gets from the ssh session. i have not figured out how to get this to be passes as "vt100" by the invoking ssh inside the powershell or command CLI on the initiating side. but if i override it on the remote shell, everything i invoke there hears that my terminal type is "vt100".
i guess coursera and udemy don't have shells and the TERM variable their sshd gets from your ssh has to be correct? sorry about my ignorance, i don't use ssh that way.
-- P Vixie