Add regex-replace highlighter to replace sections by regex

[topisani]

This highlighter is used to replace like replace-ranges does, but with regex, making it useful for syntax highlighting.

This (terrible) demonstration shows it used to wrap markup and show lambda as λ.

Other uses could be:

• Displaying operators as unicode symbols
• Hiding the *stars* around markdown tags
• Custom UI elements

The code currently duplicates a lot of the stuff in RegexHighlighter and is very experimental. It supports substituting in the contents of match groups, which are currently read with their display atoms, since that copies selection highlighting and more, giving it a much more natural feel

It has the same restrictions as replace-ranges, meaning it cannot be multiline.

There is still a lot of things to work on with highlighters, i think line folding is probably one of the top next things, but those would require some deeper work i think.

Edit: another demo (once again sorry for the colors) This is my setup of markdown and @TeddyDD's wonderful wiki. It still however has some issues. The main thing is, that when the cursor is on hidden text, the cursor is invisible, and you still have to move through the text. This is a recurring oddity, and i think there should at least be an option to skip replaced ranges when moving around. I have not yet looked into that part of the codebase, to see if it is even feasable.

With highlighter introspection in userland, this could be done there as well, however I think this is important enough to find another solution.

[mawww]

As I said on IRC, my main problem with replace-regex is that it will hide buffer text, and will lead to users asking for display based movements, which we cannot do by default if we want Kakoune's editing language to be predictible (being dependent on the currently active highlighters seems pretty bad), and would be a huge can of worm to open (all the text objects definition become much more ambiguous once we can either consider the buffer content or the displayed content).

Most use cases for replace-regex I heard of seem to be better done by a rendering system, that generates a new buffer in which some source data is "rendered" (this exists already for the :doc command, which renders asciidoc files). This has the advantage not to conflate the display and editing buffer (although one could argue that we could use two windows on the same buffer, one using replace-regex, and another not, to achieve that).

Another use case I can think of for replace-regex seems better suited to terminal/UI side, like ligatures.

[topisani]

I definately get your concern with display based movements, however, i think it is important enough to figure out a solution.

a rendering system, like for asciidoc is great, but becomes a lot more messy when it's a file you have to edit. Attempting to keep two buffers in sync then is going to give a lot of issues.

I don't have a good solution for display based movements, but i already feel like we need it for example for wrapped lines.

I guess the options are something like:

• a display prefix key which makes any movement act by display
• per-highlight options. For example, for the markdown delimiter highlighting it shouldn't move by display, since you still want to be able to edit the delimiters.

Even without display based movement, i think this is very useful. The main thing i think it lacks, is cursor highlighting on the replaced text.

Regarding ligatures, i dont think it should be done by the terminal/ui, since it will often depend on filetype and context.

With all of this, Here is what i would prefer:

• bring in the regex-replace highlighter pretty much as is
• highlight the selection on replaced text
• Don't do display-based movements for now

I think this aproach will feel a lot more natural than display-based movement anyway, since it's clear, that it is only the display that changes.

[mawww]

To give an update on that PR, I dont plan to merge it as-is, as it duplicates too much code for my taste, it should be sharing most of the code with the normal regex highlighter. I also think the replacement code should be extracted as a general regex utility (that takes a format string and a MatchResults, and generate the replaced string).

I am ok with merging the feature now that I have a clear idea of how to handle hidden text (all highlighter that possibly replace text should disable themselves on replaced ranges that are partially selected).

[topisani]

I'll take a look at implementing this better some time. Your aproach to hidden text sounds very reasonable!

[velrest]

@topisani Hey, i just stumbled upon this via #2536 while searching for a way to replace JS => arrow functions with →. Is there some new progress on this? This would be an awesome feature to have! I don't have much experience with kakoune but if there is something i can help with i would gladly give it a try.

[chaulkie]

Been subscribed to this thread for a while hoping for some progress. Would just like to add my voice and say that I love Kakoune, but this particular issue is the main thing holding me back from making any attempt at switching over from my Neovim setup. I think this feature would seriously increase Kakoune's attractiveness as a daily driver.