helix

helix copied to clipboard

I noticed that your website references creating a gui with skia/skulpin. That sounds very similar to my architecture for neovide which is a gui for neovim.

A not broadly stated goal for neovide is to eventually expose the graphical parts as a swappable front end for other text editors such as what helix looks like. I was curious if you are interested in such a collaboration. Getting a gui right is hard work (as I've found out), I think collaborating would be great for both use cases!

Are you planning on exposing a gui protocol ala neovim's? If so what would that look like/what features do you think would be useful for helix that are unique to helix? The editor looks very cool btw!

Hey! The skulpin mention was specifically after I looked at neovide, it seemed like a good fit. @norcalli wanted to experiment on a new frontend as well.

I'm a bit apprehensive on exposing a GUI protocol, they're either too vague and the client needs to re-implement a lot of the handling, or they're very specific and every client ends up the same. I don't have a solid opinion yet so we'll see. Right now the idea is that helix-view would provide most of the logic, and a frontend would just be another crate wrapping everything with a UI -- that needs some work right now though because commands are currently sitting directly in helix-term.

That makes good sense. One thing thats been on my list for a while is to split neovide into the neovim gui, and a front end renderer so that other apps could build their own rendering. Then you could integrate it as a crate without having to do all the painful font rendering and such. Let me know if something like that would be interesting for your use cases or not. In any case, if you end up going the skia route, I'm happy to help out if you run into problems. Awesome project

The more I think about it, the more I am interested in trying out something like this. You mentioned that commands are currently located in the helix-term. Are you suggesting that they would need to be moved into helix-view in order for a gui crate to consume and produce them? Are you interested in a PR which attempts that kind of refactoring?

So there's a few sets of changes necessary:

• [x] helix-view needs to be refactored to have it's own generic key/style/color structs. https://github.com/helix-editor/helix/pull/345 is a start but it depends on crossterm. Using generic structs would let us drop the crossterm dependency from helix-view.

• [ ] helix-term commands.rs need to be moved into helix-view. The problem there is that the context depends on the compositor written for the TUI, and some commands directly manipulate it by pushing in new components.

  The solution here is to make the compositor a trait so that each frontend can implement it's own. The new trait would have methods like cx.compositor.popup() so that we can manipulate UI without directly referencing the types. But I'm not sure how to handle cases like these https://github.com/helix-editor/helix/blob/fd1ae35051b57e689f6e6ef7e03c552a78f3f33a/helix-term/src/commands.rs#L3084-L3086

• [ ] There's a bit of lsp code in term/src/application.rs that would be better off shared somewhere.

With these changes helix-view would end up holding most of the code, and -term would simply be a tiny frontend implementing the compositor & components.

I'm looking into step 2 now that step 1 has merged. Let me first say back to you what I think I understand you to mean for how we might move the commands out of helix-term. Then I will propose a slightly different strategy with some benefits and drawbacks.

Gui as a component drawer

The solution here is to make the compositor a trait so that each frontend can implement it's own. The new trait would have methods like cx.compositor.popup() so that we can manipulate UI without directly referencing the types. But I'm not sure how to handle cases like these

So what I think you are suggesting is that each frontend would create a custom compositor type which implements the Compositor trait. This trait would have functions on it for each of the custom components that are implemented in the helix-term crate today. Your example is popup, but I'm guessing this would also require picker, prompt, spinner, menu etc.

Some pros for this approach:

• Custom more native UI can be implemented for each of these effectively giving more control to the gui
• The gui can implement surfaces however they like. Doesn't have to have everything on a strict grid
• This is a relatively small change assuming all of the components can be implemented as functions

Some negatives that come to mind:

• Adding new components requires implementing them in every front end (probably just term and gui, but who knows maybe multiple front ends are created)
• Guis could be very different in feel from each other. Would configs work the same in all of them? Neovim has this problem because often a config that works great in one gui is a complete mess in another
• Theres a relatively large implementation burden for the guis. Even though popup and prompt are basically just rendering text in a window of some sort, they would both have to be implemented in the front end

Gui as a render surface

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

If however more gui extensions are implemented, the rendering becomes slightly more complicated. Rather than drawing a single grid, multiple grids can be defined and drawn to. This gives the gui some more freedom to effect the size and positioning of the rendered windows, but components are still done just using that basic framework. Popups are just smaller windows which render text into them. Floating terminals are implemented on floating windows with text just the same as a dialog box.

This also separates innovation in the editor from implementation in the gui. Components can be introduced without requiring them to be implemented in every front end.

Some benefits:

• Guis are pretty simple. Basically they are just a rendering system which components can depend on to do their drawing
• Guis don't need to be updated to add new components
• Many guis can be created which benefit from upstream updates and changes. Can start to build an ecosystem if there is desire to

Some drawbacks:

• Guis can only innovate so far as the interface allows. It would require some creativity to let guis do special rendering for a given type of component -- maybe this can be fixed by giving each rendered window a "purpose" flag which lets the gui special case a certain component
• Pretty significantly different approach to the current architecture. Would maybe require some intermediate steps
• Likely locks guis into a grid based interface. I personally feel this is nice because it simplifies design and such, but I can understand why others would prefer more flexibility.

Thoughts

This has been a bit ramble-y. I hope my point was at least somewhat understandable. I'm interested in your thoughts.

@Kethku If anyone were to take on GUI probably you are the one that is most familiar with it since you did neovide. My thought is that we may want it as a render surface but with custom components. It is useful in doom emacs where you can draw something not available in the terminal.

Like you can see on the left, green for new lines, yellow for modified line. And one more you can see is a single red dash for removed lines, I don't think we can easily render these in terminal.

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

There is also another possibility, getting neovide to have helix backend? Not sure how hard is that but from current viewpoint, helix does not have anything, but yeah neovide currently seemed to only be drawing the same thing as terminal, in some places it would be good to have certain way of drawing which is not possible in terminal.

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

To be honest, a good terminal emulator inside of helix sounds wonderful. I personally rarely dip into a terminal emulator outside of the one contained within neovim. But I think thats a large can of worms that doesn't need to happen now. What I'm interested in helping with is just making sure the design of helix is conducive to gui front ends in the long run even if its not ready yet.

There is also another possibility, getting neovide to have helix backend? Not sure how hard is that but from current viewpoint, helix does not have anything, but yeah neovide currently seemed to only be drawing the same thing as terminal, in some places it would be good to have certain way of drawing which is not possible in terminal.

I'm definitely thinking about ways to make more full fledged guis using Neovide's rendering system. What I would LOVE is to split Neovide in half so that the rendering part is just a crate which provides channels for sending draw commands and receiving input/events from the window. Then the renderer would handle everything else. Integrating that crate with an editor backend would just require writing some glue.

Neovide isn't there yet because there are lots of neovim specific-isms that would need to be moved around, but I think it would be valuable to do because features built for one editor backend would likely be useful for others.

To be honest though its kinda a pipe dream at the moment. What I was thinking in the short run is to just recreate something like neovide's rendering system for helix specifically in order to get off the ground, and then think about unifying the two later if there is interest. That way I don't impose further constraints on this project that don't really benefit this project long term

Like you can see on the left, green for new lines, yellow for modified line. And one more you can see is a single red dash for removed lines, I don't think we can easily render these in terminal.

This is doable using box drawing characters (I imagine emacs is using the same thing). https://github.com/archseer/snowflake/blob/e616d7cb24d85cf6b17b77c3e0cb1ef6c4212414/profiles/develop/neovim/init.vim#L294-L299

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

I was a bit undecided between the two approaches. While the neovim approach works, it leads to most frontends looking very similar. (For example, while I'm aware of Neovide and looked at it's source, I haven't actually used it yet because I didn't have enough reasons to switch.)

A character grid is a good start but is limiting if we want to experiment with inline decorations (see the CodeMirror 6 examples). These might be variable sized or won't fit perfectly in the character grid. Another example would be rust-analyzer's clickable code action links that are embedded into the view.

We're also limited by the terminal in that it's hard to render 0 width multiple selections (cursors) so selections are always 1-width or more (see https://github.com/helix-editor/helix/issues/362 for more context).

I'm not even sure if the component drawer approach is flexible enough, since we might want to use completely different component types on different frontends. I guess we could allow for frontends to provide their own sets of commands that would augment the defaults with different UI. Maybe it's okay for the frontends to share the same core/view but end up being slightly different (similar to scintilla, or codemirror embedding)

I've been keeping the built-in component list short: https://github.com/helix-editor/helix/tree/master/helix-term/src/ui

Feel free to join us on Matrix to discuss further! (https://matrix.to/#/#helix-community:matrix.org, make sure to join #helix-editor:matrix.org if you can't see it on the list)

---

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

We won't only be doing a GUI, a helix-term is still going to be the primary implementation. (Well, one of two, I'd like to have the skulpin implementation as "officially supported" too). Down the road we'd want to embed a terminal, but that's quite a bit of work.

I'd also like to get @cessen's opinion here as well

I was a bit undecided between the two approaches. While the neovim approach works, it leads to most frontends looking very similar. (For example, while I'm aware of Neovide and looked at it's source, I haven't actually used it yet because I didn't have enough reasons to switch.)

This is a very reasonable critique of Neovim's approach. To me the difference between the two depends almost entirely on if you want to maintain the canonical gui for helix or if you want there to be many different guis which use helix as the backend. If you imagine the main helix gui is basically the only one people will use, then I agree it makes a lot of sense to have component aware logic in the gui because it can tweak the experience to feel just right. There would still be reimplimentation of logic between the terminal and gui front ends, but hopefully that wouldn't be too bad.

However if you want to enable many different front ends, I think that strategy falls apart a bit because any time you want to tweak the behavior of a given component, you have to convince all the front ends to also update their implementations. Its harder to innovate like that.

I'm definitely biased given that neovim is what I've spent the most time with, but I think the vision of a gui api which can be implemented very simply as just a grid, but then augmented with optional extensions if the gui developer wants to customize a given type of behavior is beautiful. It means that if somebody wants to make a gui which is just simple tweaks off the terminal experience, they can pretty easily. And if they want to make something that completely reworks the user experience, thats possible too with a little more effort.

A middle ground

If you are interested in building an ecosystem where multiple groups can implement guis if they want to, then it seems to me that a great way to do this would be to implement the grid based approach. Basically move all the component logic into helix-view, and tern helix-term into a super thin rendering surface. Basically just cursor moves and text on a grid. Then each of the components could be implemented as extensions optionally. If the gui wants to customize the dialog box, theres a trait they can implement for their gui which takes over drawing of dialog boxes completely. And a different trait for taking over markdown renders etc etc etc

This solution would be very similar to neovim's extension model where guis can re implement the tabline or message rendering if they want to, but the difference is that helix's implementation would be implemented with components as the basis from the start. Rather than hacking in extensions for features editors would like, helix has the benefit of starting from scratch and thinking about components as first class extension points.

---

All that said, these are just some quick thoughts. I'm happy to help out whichever way you end up going.

I'll pin this issue because it seems to me it could use more visibility. We currently pin two issues out of the maximum of three, so I went ahead.

A character grid is a good start but is limiting if we want to experiment with inline decorations (see the CodeMirror 6 examples). These might be variable sized or won't fit perfectly in the character grid. Another example would be rust-analyzer's clickable code action links that are embedded into the view.

I think for the document text itself, we'll want to stick with a character grid.

The short justification is: it allows the editor back-end to control text layout. The architecture I'm imagining is basically: the back-end is given the grid dimensions, it computes where to place each element within that grid and passes that information to the front-end, and then the front-end renders the described grid (wherever and however it wants). This keeps the flow of data really simple, and especially avoids the front-end needing to send text layout information back to the back-end for things like cursor movement commands (...which also applies to off-screen text thanks to multiple cursors).

In other words, it lets us keep all of the core commands in Helix independent of the front-end. Conversely, if text layout is computed by the front-end (which I think would be more-or-less required if we forego a grid), then the back-end will have to query the front-end when executing any commands that involve text positioning.

So the grid would essentially act as a text layout API between the front-end and back-end, letting us keep as much code as possible compartmentalized in the back-end. (And as a bonus, that also avoids having to rewrite the layout code for each front-end.)

but is limiting if we want to experiment with inline decorations

I don't think it's quite as limiting as you might think. The back-end would control where things are placed on the grid, but the front-end decides how to render them. So, for example, an inline element could be rendered as a fancy, nice-looking button by the GUI front-end, as long as it fits in the grid area allocated to it by the back-end.

And the front end could provide a list of "size factors" for the various kinds of inline elements, so that the back-end can ensure to allocate enough grid spaces for each kind of element.

We're also limited by the terminal in that it's hard to render 0 width multiple selections (cursors) so selections are always 1-width or more (see #362 for more context).

I have some ideas about this, that I think will be easier to demonstrate with a prototype once I've completed work on #362. Although I realize that might not be especially convincing right now, ha ha.

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

After re-reading this, I realized I might want to clarify: I'm only talking about individual documents, not the entire GUI layout, when I'm discussing the character grid. As far as I'm concerned, the front-end can render everything else however it wants. I'm not totally sure how we would present an API for the remaining commands (e.g. file finder, etc.), but I'm not too worried about us being able to work it out.

Like you can see on the left, green for new lines, yellow for modified line. And one more you can see is a single red dash for removed lines, I don't think we can easily render these in terminal.

This is doable using box drawing characters (I imagine emacs is using the same thing). https://github.com/archseer/snowflake/blob/e616d7cb24d85cf6b17b77c3e0cb1ef6c4212414/profiles/develop/neovim/init.vim#L294-L299

FWIW, emacs can actually draw bitmaps on its fringe and in modeline, it's not restricted to whatever unicode supports: https://www.gnu.org/software/emacs/manual/html_node/elisp/Fringe-Bitmaps.html

Of course, it only works in GUI mode. I'm really interested in a proper gui for the editor as well, but I don't have anything particularly productive to add to the discussion, mostly subscribing for the thread with this emacs remark :)

Of course it would be cool to have a gui that supports custom shaders and all that crazy stuff to fully utilize its hardware acceleration capabilities (if we take neovide as a mental base, I mean)

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

After re-reading this, I realized I might want to clarify: I'm only talking about individual documents, not the entire GUI layout, when I'm discussing the character grid. As far as I'm concerned, the front-end can render everything else however it wants. I'm not totally sure how we would present an API for the remaining commands (e.g. file finder, etc.), but I'm not too worried about us being able to work it out.

Just wanting to elaborate a bit. I'm not super attached to one option or another, but including in the the drawing api some concept of window/control layout has two main advantages in my mind

1. The terminal and gui act more similarly. If the window positions are defined in terms of a base grid/position in other windows for things like popups, then people won't have as much adjustment if they want to try the gui.
2. It allows for progressive implementation of a gui. This may be a non goal, but I think its valuable that Neovim lets you create a functional gui in just a couple days by just implementing the basic grid rendering. Then a given front end could progressively add features by taking over more and more of the process. I think thats a pretty cool and efficient aspect of their gui extension model.

But I do think its a decision to be made and pushes more of the logic into the editor core than may be preferable. And it does limit what a gui can customize somewhat, so I could see value in both options.

To get started with designing client-server model and splitting out UI it might be convenient to make a simple web-based GUI for browser, as a temporary GUI without harder to do stuff like GPU rendering. Then a proper GUI can be made based on well-developed and debugged concepts, but natively implemented using wgpu or higher level toolkits like iced and kas.

Overall, this seems like a really dependent on #312. GUI should work through network easily, while backing server keeps track of changes, language features etc. Plugins might also be available both for server and client, depending on their actions. While working locally without inviting others for pair coding, backend might be communicating with GUI via stdio.

helix-term commands.rs need to be moved into helix-view. The problem there is that the context depends on the compositor written for the TUI, and some commands directly manipulate it by pushing in new components. The solution here is to make the compositor a trait so that each frontend can implement it's own. The new trait would have methods like cx.compositor.popup() so that we can manipulate UI without directly referencing the types. But I'm not sure how to handle cases like these https://github.com/helix-editor/helix/blob/fd1ae35051b57e689f6e6ef7e03c552a78f3f33a/helix-term/src/commands.rs#L3084-L3086

This will solve #507, I'm currently working on this.

I'm making some progress on https://github.com/helix-editor/helix/pull/711 again.

The goal is to move all commands out of helix-term into helix-view (or helix-commands).

I tried moving the Compositor into helix-view then splitting Component into an abstract Component trait that handles input + Render trait that's frontend specific (then components + compositor would be generic, and each component would get a render implementation per frontend). That got rather messy though since there was no easy way for components owned by the compositor to be Component + my_frontend::Render without a ton of generics on all the commands.

Instead, I'm currently trying a different approach that uses an events return value on the command context. This would allow commands to signal intent without being too specific about the UI being used to handle it so each frontend could be flexible in dealing with these events.

I'm not sure how granular this should be yet:

UI events

picker(vec<symbols>, on_select_fn, etc.)
popup(id:lsp::doc, text)

vs

metadata events

lsp::symbols(vec<symbols>, on_select_fn, ...)
lsp::doc(text)

Since helix is done in rust, would Tauri tauri-apps/tauri be a good candidate to offer a GUI ? Has it been looked at, dismissed ?

Seems to be more heavyweight than necessary. Functionality required for Helix frontend can be implemented without any web technologies. Might still be a good thing for diversity of frontends and, possibly, as initial impl.

I'm not interested in webview-like frameworks, currently experimenting with doing the rendering in femtovg / webgpu. I also evaluated piet but looks like it'll be a while before piet-gpu is ready.

Any new updates about this?

Character grid rendering rules out elastic tabstops with modern proportional fonts, which I would dearly love to have in a full-featured code editor.

I would also love to collaborate on helix GUIs (native, WASM) for the editor in my "revision control for everyone" project, but proportional fonts are a must-have so character grid rendering would rule out using helix.

What is the current state? It would be very nice if someone created something like a roadmap of what needs to be implemented and what has already been implemented. I cannot start from scratch myself as I am quite new in rust.

The relationship between helix-term and helix-view needs to be refined so that only terminal-specific functionality is in helix-term. Afterwards, presumably it would be to implement a helix-view backend for neovide. It's not a high priority at the moment.

What does this have to do with a gui?

What does this have to do with a gui?

I'm confused, was the comment this was in response to deleted?

I deleted my comment. cause seem to no relate to gui. I'm sorry.

Another argument against using a character grid for documents is that it rather rules out various forms of inlays - I've seen these used very well by e.g. Error Lens in VS Code, with the font set to be significantly smaller than the source text, making it immediately distinct to the user.