Weird control characters when pasting from windows clipboard to WSL2

[0anton]

Windows Terminal version

1.23.10353.0

Windows build number

10.0.19045.0

Other Software

cat, vim 9.0 (inside WSL2)

Steps to reproduce

Since recent time I encounter weird control character pasted before my actuall text, when I try to paste by clipboard into WSL terminal. It is reproduceable in vim and with simple cut:

Notice this unexpected characters ^[ at the beginning of the string:

 $ cat > test-file.json
^[{
  "pubsub_msg": "\"{\\\"kind\\\": \\\"storage#object\\\", \\\"id\\\":

cat > test-file.json

Right-mouse paste works fine, no extra characted is inserted:

[{
  "pubsub_msg[{
": "\"{\\\"kind\\\": \\\"storage#object\\\", 

The behavior is both reproduceable with Ctrl-V and with Ctrl-Shift-V.

I tried to follow the advice of https://github.com/microsoft/terminal/issues/10572 and put into .inputrc lines:

set enable-bracketed-paste off

but it does not help...

The problem only appears in Windows Terminal, but not in wsl.exe for example.

The problem does not affect other shells, like PowerShell or CMD.EXE.

Interestingly, if the content is not formatted like hello world, it is being pasted withtout weird control character:

$ cat > /dev/null

hello world

Does anybody knows what is this problem? It is very annoying to manually remove this characters from vim buffers manually after every paste...

[0anton]

Interestingly, the .inputrc's set enable-bracketed-paste 0 still works in normal release:

Windows Terminal
Version: 1.21.10351.0

But has not effect in preview:

Windows Terminal Preview
Version: 1.23.10353.0

So, I "fixed" it for myself by reverting from the preview back to normal. Still want to discuss it, before the problem hits the stable channel.

UPDATE

Unfortunately above settings only fixes pastes to vim (WSL2), but not pastes to cat (readline?) So, it is still an issue to me...

[carlos-zamora]

Thanks for filing! Can you reproduce the issue with Debug Tap enabled and share the output /? Instructions for how to use Debug tap can be found here: https://github.com/microsoft/terminal/wiki/Troubleshooting-Tips#enabling-the-debug-tap

This should help us debug this issue further 😊

[0anton]

@carlos-zamora thanks for suggesting! cool troubleshooting method! I hope it will help to understand a problem.

So, I prepared a simple json string { "a": 1 } and pasting it to vim buffer and to cut stdin:

1. Paste to vim -> correct

d057039:~$ vi a <Enter><Ctrl+V>
{ "a": 1 }

Key logger:

~␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[m"a"␣[New]␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣0,0-1␣␣␣␣␣␣␣␣␣All␣␛[1;3H␛[13;28;13;0;32;1_␛[73;23;105;1;32;1_␛[?25l␛[1m␛[80;1H--␣INSERT␣--␛[22m␛[36C0,1␣␣␣␣␣␣␣␣␣␣␣All␣␛[1;3H␛[?25h␛[73;23;105;0;32;1_␛[17;29;0;1;40;1_␛[200~{␣"a":␣1␣}␛[201~{␣"a":␣1␣}␛[?25l␛[46m␛[1;3H{␛[8C}␛[m␛[80;49H1,11␛[1;13H␛[?25h␛[86;47;22;0;40;1_␛[17;29;0;0;32;1_␛[O

2. Paste to cat stdin -> incorrect - enters ^[ instead of {:

d057039:~$ cat > /dev/null <Enter><Ctrl+V>
^[ "a": 1 }

Key logger:

␛[K␍␊
␛[K␛[1;12H␛[?25h␛[76;38;12;0;40;1_␛[17;29;0;0;32;1_␛[38;72;0;1;288;1_cat␣>␣/dev/null␛[38;72;0;0;288;1_␛[13;28;13;1;32;1_␍␊
␛[13;28;13;0;32;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_␛[17;29;0;1;40;1_{␣"a":␣1␣}^[␣"a":␣1␣}␛[86;47;22;0;40;1_␛[17;29;0;0;32;1_␛[O

Now the same test in the same WSL2 console but using

Windows Terminal Preview
Version: 1.23.10353.0

d057039:~$ vi a <Enter><Ctrl+V>
^[ "a": 1 }

Logger:

␛[m␛[80;49H0,0-1␛[9CAll␛[1;3H␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?12$p␛[?12;2$y␛[?25l␛[?25h␛[?25l␛[?25h␛[13;28;13;0;32;1_␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[?25l␛[?25h␛[73;23;105;1;32;1_␛[?25l␛[80;1H␛[1m--␣INSERT␣--␛[m␛[80;49H␛[K␛[80;49H0,1␛[11CAll␛[1;3H␛[?25h␛[73;23;105;0;32;1_␛[17;29;0;1;40;1_␛[200~{␣"a":␣1␣}␛[201~␛[?25l␛[38;5;81m^[␛[m␣"a":␣1␣}␛[?25h␛[?25l␛[80;49H1,11-12␛[1;14H␛[?25h␛[17;29;0;0;32;1_␛[86;47;118;0;32;1_␛[O

As you can see the VIM is broken here (exactly same WSL2 terminal as above)

wsl2.exe provides clean input (right mouse paste)

disabling bracketed paste does not work:

$ bind 'set enable-bracketed-paste off'
d057039:~$ cat > /dev/null
^[ "a": 1 }^C

what could be the reason?

[DHowett]

Thanks for capturing those! I'm surprised to see that the data is getting yeeted into the input buffer correctly.

␛[17;29;0;1;40;1_␛[200~{␣"a":␣1␣}␛[201~
[Ctrl Down]      BP+   Content   BP-

Would you mind pasting into showkey -a inside WSL2? It should be in the kbd package (Debian, Ubuntu) or your distribution's equivalent.

I suspect the issue will end up being somewhere in the unchartable middle, in one of our console app API translation layers.

[0anton]

@DHowett hope it helps, see below. I've been using Windows Terminals without this weird paste issue.

$ showkey -a

Press any keys - Ctrl-D will terminate this program                                                                                                                                                                                                                                                                     ^["a": 1}        27 0033 0x1b
         34 0042 0x22
         97 0141 0x61
         34 0042 0x22
         58 0072 0x3a
         32 0040 0x20
         49 0061 0x31
        125 0175 0x7d

Be aware that above the pasted input is for some reason only correctly shown in the edit mode, I'm cutting it out separately:

^["a": 1}        

[PankajBhojwani]

Thank you for logging! We suspect we are doing something incorrect when we receive the "opening bracket" character (specifically ctrl+shift+bracket looks the same to us as ctrl+bracket).

Potential bodgy hackfix: when we dispatch a shortcut action, should we tell conpty that the user released special keys? (this may not actually work but needs to be investigated)

[0anton]

@PankajBhojwani thanks for picking this up.

I‘d like to add another observation, which may help narrowing down the problem even more.

I‘ve noticed that the problem is only appearing if I paste a JSON string. It looks like some layer don‘t like the combination of bracketed paste sequence and the opening curly bracket { of the JSON string.

[og900aero]

I have exactly the same problem with { } [ ] characters.

[og900aero]

There is another problem character: ~

[0anton]

Very annoying. Affects all JSON pastes into Terminal to vim and cat. Does anybody is aware of a workaround?

[DHowett]

Does anybody is aware of a workaround?

Does right-click paste exhibit the same problem? Sorry, I can't remember if we'd asked this before but I didn't see it in the thread.

[0anton]

@DHowett

Right-click indeed paste correctly:

$ cat > a
^[ "a": 1 }   <-- Ctrl-V, broken
{ "a": 1 }    <-- Right-Click, normal

$ vi
1 ^[ "a": 1 }    <-- Ctrl-V, broken
2 { "a": 1 }     <-- Right-Click, normal

Weirdly, if I try to remap Ctrl-V to mouse right click with AutoHotKey I still get ^[ "a": 1 } in Terminal...

AHK snippet:

#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
^v::Click right
#IfWinActive

[piradata]

other strange thing, "/" is being translated as "^_" if its the first character from the copy

[j4james]

For the record, I think this is the same issue as #17264. It's triggered by keyboard layouts where the [ character is generated with an AltGr key combination. For example, on a German keyboard layout [ is produced by AltGr+8. You also need to be pasting with a Ctrl key shortcut - a mouse paste shouldn't trigger it.

[piradata]

if i past with mouse right click, that still generates the wrong character..

[j4james]

@piradata That's probably a different issue then. But it might help knowing what keyboard layout you're using, what terminal version, and what shell you're pasting into.

[piradata]

@piradata That's probably a different issue then. But it might help knowing what keyboard layout you're using, what terminal version, and what shell you're pasting into.

shell: zsh 5.9 (x86_64-ubuntu-linux-gnu) terminal version: 1.22.11751.0 keyboard layout: ABNT2 (BR)

And I was wrong, sorry, just tested right clicking with the mouse and it pastes ok. The problem only happens with ctrl + shift + v

[j4james]

@piradata OK, that's the same issue then. On your keyboard, a / can be typed as AltrGr+Q, so that's what the terminal simulates typing when it's pasted. And when the Ctrl key is held down at the same time, an AltGr key combination like that can get misinterpreted as Ctrl+AltGr+Q, which produces the^_ that you're seeing. This will hopefully be fixed by PR #19083.

[0anton]

does anybody know how to test the fix? The issue is still bothering me on the preview version of the Terminal.

Windows Terminal Preview
Version: 1.23.11752.0

Very annoying - after every paste of JSON content I need to manually fix the first corrupted character.

[DHowett]

does anybody know how to test the fix?

The canary build!

[0anton]

Thanks @DHowett

I confirm the Windows Canary works for me - it fixed JSON paste problem (Ctrl-Shift-V)

Beta and Current channel (bug present):

Canary (bug fixed):

Tested with:

{
  "a": 1
}

[DHowett]

Thanks for confirming! This change is being released today as part of a 1.22 and 1.23 patch update. :)

[DHowett]

Well, it missed the boat for 1.23. It'll be in the next 1.23 build. It's in 1.22 though!

[og900aero]

I installed new terminal preview 1.23.12102.0, and the problem still persists.

When I copy the following code to the new windows terminal preview:

[custom.ansible]
description = "Ansible version"
command = "echo -n $(ansible --version 2>/dev/null | grep ^ansible | awk '{ print $2 }') $(which ansible | sed -e s@$HOME/@@ -e s@/bin/ansible@@ -e s@/.virtual_env@@)
when = """ test -n "${ANSIBLE_HOME}" """
format = "[$symbol$output]($style)"

And I get this:

^[custom.ansible]
description = "Ansible version"
command = "echo -n $(ansible --version 2>/dev/null | grep ^ansible | awk '{ print $2 }') $(which ansible | sed -e s@$HOME/@@ -e s@/bin/ansible@@ -e s@/.virtual_env@@)
when = """ test -n "${ANSIBLE_HOME}" """
format = "[$symbol$output]($style)"

[lhecker]

To quote Dustin above:

It'll be in the next 1.23 build.