Feature Request: Bracketed Paste Mode

[lentinj]

Would you be interested in support for bracketed paste in ne? I mostly remember to disable auto-indent before middle-button-pasting, but always...

I can just about do it in configuration with the following in .ne/.keys:

SEQ  "\x1b[200~"  1FE
SEQ  "\x1b[201~"  1FF
KEY  1FE          Macro bracketed-paste-push
KEY  1FF          PopPrefs

The following in .ne/bracketed-paste-push:

PushPrefs
AutoIndent 0

...and then defining ne() { printf '\x1b[?2004h' ; /usr/bin/ne "$@" ; printf '\x1b[?2004l'; }.

This works in xterm at least, a patch to build similar behaviour into ne looks easy enough, but it's not obvious whether sending the enable / disable commands indiscriminately (e.g. in set_terminal_modes) would be okay, or if it ought to be behind a preference.

[utoddl]

Thanks for the suggestion, @lentinj . It's sparked some interesting discussion around here. I see no reason not to support bracketed paste. It solves several problems, one of which - the AutoIndent issue in particular - is a frequent annoyance.

[lentinj]

Good to hear! Another nicety that native support could have is not redrawing the screen until the end, like a regular ne paste operation.

It possibly ties into the mailing list discussion about where the cursor ends up afterwards. I've personally grown accustomed to ctrl-v being at the start of the paste, middle-button-paste going to the end. It'd be slightly sad to lose that as a result, but if there's a preference to where the cursor ends up that affects both, that's probably a more reasonable long-term behaviour IMO.

The most obvious gap in support thus far seems to be GNU screen, but that looks like it's closing: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1019576

[utoddl]

@lentinj , I've geen investigating issues around supporting bracketed paste, and I'm coming up short on a fundamental question. There is no supported way for termcap/terminfo to indicate whether a given terminal supports bracketed pastes. So how does a well-behaved program know whether it's safe to request bracketed pastes? There's the "off by default/let the user decide" option, which seems rather undiscoverable, and there's the "on by default and hope it doesn't trash the user experience" option. I'd appreciate some guidance. Thanks.

[lentinj]

This was the key part I wasn't sure about either.

This page is a pretty convenient record of support, and I've rummaged a bit to find out how bracketed paste works in a few:

emacs

Openly admits it will just turn it on and hope they don't mess up any terminals:

https://github.com/emacs-mirror/emacs/blob/93b9cf41846b40cd050b56f6e83b590330be255e/lisp/term/xterm.el#L908-L910

vi

I didn't quite follow the code, but seems like it's enabled by default, but you can disable it by clearing the variable t_BE (by default containing the escape code for enabling bracketed-paste):

https://github.com/vim/vim/blob/df5320c439e9a7f7bf1ebff3cb455d45e223547a/runtime/doc/term.txt#L106-L113

There's certainly no evidence of turning it on/off for various scenarios in the original patch.

bash / readline

On by default (with the slightly ominous "for now"...):

https://github.com/bminor/bash/blob/f3a35a2d601a55f337f8ca02a541f8c033682247/lib/readline/rlprivate.h#L315-L317

...but has allowances for obvious terminals that won't (BRACKETED_PASTE_DEFAULT is the default for _rl_enable_bracketed_paste and turns on/off the writing of the escape code)

https://github.com/bminor/bash/blob/f3a35a2d601a55f337f8ca02a541f8c033682247/lib/readline/terminal.c#L604-L607

zsh

Like vim, has an option with the escape codes to on/off bracketed space, set by default, and tells you to clear them to turn it off:

https://github.com/zsh-users/zsh/blob/00d20ed15e18f5af682f0daec140d6b8383c479a/README#L389-L394

---

So the general consensus seems to be "on by default and hope it doesn't trash the user experience", bar obvious cases like ne --ansi. This seems pretty terrible, but I guess if most applications assume they won't break terminals by outputting \x1b[?2004h where it's not expected, including the shell that's running ne, then the risk of breakage isn't that high in practice?

[utoddl]

If you're still interested, have a look at the bracketed-paste branch.

ne has several things that you may want to act differently with bracketed paste besides AutoIndent. As it is, the bracketed-paste branch lets you control AutoIndent, Tabs (whether tabs remain as tabs or get converted to spaces), WordWrap, and an Atomic option (see Help AtomicUndo) which lets you revert the entire paste with a single Undo command vs. one Undo for each character in the bracketed paste. The defaults have AutoIndent and WordWrap off and Tabs and Atomic on.

It's fairly clearly documented in the BracketedPaste command reference, but there should be some discussion elsewhere in the ne.texinfo document. I wanted to get this out there, and the docs always take longer to write than the code.

There is another way to handle BracketedPaste, and that is to have two macros, one of which gets run at the beginning of a bracketed paste and the other at the end. The former would do a PushPrefs and tweak whatever needed tweaking, while the latter would do a PopPrefs to return all options back to their former settings. That may be more flexible, but I'm disinclined to bring in additional files which may or may not exist, etc.

I look forward to your comments.

[lentinj]

I've still not had a chance to try it locally yet, apologies, but reading through the patch looks like my dreams would come true!

Whilst it would certainly work for me, I'm not a huge fan of the bitmask option. I'd suspect either users would either not care about the bitmask minutiae and want a "do something sensible", option, or would find the bitmask too restrictive and want actual macros. Would something along these lines work:

• BracketedPaste 0: Disable support for bracketed paste
• BracketedPaste 1: Enable "default" support for bracketed paste (i.e. the equivalent of BracketedPaste 13 today)
• BracketedPaste bracketon bracketoff: Enable "custom" support for bracketed paste, run the 2 macros at start/end of bracketed paste.

[utoddl]

I've been warming to that approach as well: give a "probably what you want" sane default, and an easy way to customize with macros. I especially like the options you spelled out above, particularly because it lets users have any number of macro pairs for different use cases and easily switch between them.

I failed to mention before that there's a --no-bpaste cli option if for some reason you want to start ne w/o bracketed paste support. By default it's enabled, so you hardly ever need to think about it.

We do the bit-mask thing on AutoMatchBracket so there is some precedence for it. That doesn't make it a good idea though. Specifying macro names for BracketedPaste is much better.

[utoddl]

The bracketed-paste branch has been completely reworked to use macros as described above. Please give it a test drive at your convenience. (Still needs more doc updates I suppose.) If you don't do anything different from usual, it just does the Right Thing when pasting from the system clipboard - i.e. it turns off AutoIndent.

[utoddl]

I've merged the bracketed-paste branch into master. If a user doesn't change the defaults, about the only difference she might notice is that pastes from the system clipboard no longer get AutoIndented. (They also become atomic from an undo perspective.) It works as described above. There was some minor trailing whitespace clean-ups -- in the code, not UI related. And one for-real bug in the Play implementation, but nothing anyone would see unless they were running a macro containing a Play command -- macros within macros basically. I doubt anybody ever hit it.

Closing this issue/feature request. Thanks for bringing this up. It's a genuine improvement.

[lentinj]

Trying this (better late than never) with NE_DEBUG=1, moving my .ne directory out the way, I'm seeing:

ne: keys.c:546: get_key_code: Assertion `key[cur_key].code < NUM_KEYS' failed.

Bumping NUM_KEYS to (256+256+3) does the trick, then bracketed paste works as you'd expect. This seems to make sense, as NE_KEY_BPASTE_END is 0x202 == NUM_KEYS. But why this wasn't a problem for you I'm not sure.

Tried under Debian's xterm (TERM=xterm) and Debian's PuTTY FWIW.

[utoddl]

Probably off the other way, i.e. the old range was 0x000..0x1ff, and the two new NE_KEY_BPASTE_BEGIN and NE_KEY_BPASTE_END key codes were supposed to be 0x200 and 0x201 respectively. The fix you used doesn't hurt anything though. I'll double-check and get that fixed. Thanks.

[utoddl]

Those NE_KEY_BPASTE_BEGIN and NE_KEY_BPASTE_END constants are fixed now; also fixed a refresh issue during macro execution, and a misplaced parenthesis in the assert_buffer() macro.

[lentinj]

Excellent, thank you so much!