Improve pasting in REPL: don't just ignore `gap>` prompt but also output

[fingolfin]

GAP has a nice feature where copy&pasting e.g. this into a GAP prompt works:

gap> 1+1;

GAP will detect and skip the gap> prompt. Handy! It also works with continuation prompts, i.e. the following can also be pasted and works correctly:

gap> 1+
> 1;

The Julia REPL has a similar feature, but it goes one step further: when it detect that you are pasting code with prompts, it ignore outputs. Translated to GAP, that means: it shoudl be possible to copy&paste this:

gap> 1+1;
2
gap> 1+
> 1;
2

and the result would look identical. But in practice, instead we get this:

gap> 1+1;
2
gap> 2
> gap> 1+
Syntax error: ; expected
gap> 1+
^^^
> 1;
gap> 2
>

That's because it also tried to interpret the outputs as inputs. Then after the first 2 it was looking for a semicolon, and then got completely confused.

So basically after an initial gap> , assume that any line not starting with gap> or > is output and should be ignored.

[ChrisJefferson]

If anyone is interested in looking into this, an important readline option is "enable-bracketed-paste" -- the GAP code currently sets this to "off". when this is "on", then GAP can detect a whole paste being made as a single block, which will let you spot a single pasted-in block and act appropriately. It's currently set to off in sysfiles.c

Note just turning that option on will break GAP if you do a multiline paste, as the whole multiline segments gets treated as a single line.

[lispandfound]

This issue seems like a good one for me to try and solve as a starter for contributing to GAP. I have looked into @ChrisJefferson's suggestion of the bracketed paste mode in readline, and I like the idea but it seems more complicated than at first glance. For reference, I have also looked at Julia to see how it implements things.

Julia has an advantage to GAP in that it appears to implement it's own readline library instead of using GNU readline as GAP does. Because of this, it receives the pasted text raw, and this includes the ANSI escape code ESC [ 200 ~ signifying the beginning of a bracketed-paste block. After seeing this sequence, it then strips the prompts on each line of the pasted code and edits the line buffer. What the user sees is the pasted code magically losing the prompts and output. The relevant block is found in REPL.jl. This is also why typing (not pasting) the text "julia> 5" will not remove the prompt "julia>" at the start, because Julia is not binding to space like GAP does in cmdleditx.g.

I am still getting my head around how GNU readline handles bracketed paste, but by reading the manual I think I have some idea. GNU readline sits in-between the user and what GAP receives as input and strips out the bracketed paste sequences. Text inserted via paste when bracketed-paste is enabled is placed into the line buffer without calling GAP_rl_func according to the GNU readline manual - as if every key is bound to self-insert for all pasted text. It also activates the region and sets the mark to the beginning of the pasted block, and the point to the end of the pasted block (which is why text appears to be highlighted when pasting into a readline prompt with this enabled).

All this is to say that, because of the way this bracketed text is handled by GNU readline, interecepting a key and rewriting the line buffer is not possible with bracketed paste mode enabled. I believe this means it would be very hard to do the sort of magic rewriting Julia does with bracketed paste using GNU readline.

If we instead leave bracketed paste off then the behaviour is like normal, i.e. pasted text is handled as if the user typed every character manually. However, then we have to guess when the line we receive is pasted user output or the user continuing on with their session. This is easy for examples where the prompt and the output are in one-to-one correspondence like

gap> 1 + 1;
2

But harder if the command in the pasted block had multiline output like

gap> Display("for\nexample");
for
example

The compromise that seems most sensible to me is to enable bracketed-paste and preprocess the pasted code before it is evaluated. Then we do not have to guess where the output ends. This comes at the cost of changing the UX of the REPL a little. So instead of pasting "1 + 1;\n2 + 2;" and getting

gap> 1 + 1;
2
gap> 2 + 2;
4
gap>

You get

gap> 1 + 1;
2 + 2;
2
4
gap>

Personally I prefer the second behaviour anyhow as it means that the whole block is saved in history and can be rerun at once as opposed to manually recalling "1 + 1;" and "2 + 2;".