helix icon indicating copy to clipboard operation
helix copied to clipboard

Implement Kakoune-style client-server model for sharing buffers across windows in different terminals

Open malte-v opened this issue 4 years ago • 46 comments

One thing I really like about Kakoune is that it lets me use my X11 window manager for window management (duh) while still sharing buffers across windows. Kakoune achieves this using a sort of client-server model where one Kakoune process can attach to another one running in a different terminal, so effectively multiple terminals are controlled by a single editor instance. You could alternatively use tmux if you want the "vim experience", but I really enjoy not having to think about two layers of window management (editor windows/desktop windows). One other nice consequence is that you can arbitrarily mix windows containing editor buffers and windows belonging to other (graphical) applications.

What are your thoughts?

malte-v avatar Jun 19 '21 09:06 malte-v

I guess you could also keep the built-in window splitting features if people want to have that. It would just make things more complicated since one would have to distinguish between windows from different terminals and window splits.

malte-v avatar Jun 19 '21 09:06 malte-v

I think kakoune approach is nice for people who can do many terminals (like the time when I was using a tiling vm). Otherwise it doesn't help much. I still think it would be nice to have it, not sure how to implement it though. Wonder if @archseer have any thoughts on this.

pickfire avatar Jun 19 '21 13:06 pickfire

I'm against removing the built-in splitting features, as I expect that to be my primary workflow. I don't really view it as two layers of window management. (And come to think of it, I'm now remembering that this is one of the reasons I didn't dive more into kakoune, despite it otherwise seeming really cool.)

As for client/server capabilities, I'm pretty indifferent. I certainly wouldn't try to stand in the way of such a feature, but don't consider it important myself.

cessen avatar Jun 19 '21 16:06 cessen

I don't really use the splitting feature because i3 does the job for me, but I think it can be useful to some people. However, I really like the client/server architecture of kakoune allowing me to open multiple windows when working on the same instance (very useful when working using multiple screens). That's not the only approach though: in (doom) emacs I used multiple windows but I'm not sure it was client/server architecture.

CBenoit avatar Jun 19 '21 20:06 CBenoit

I'm against removing the built-in splitting features, as I expect that to be my primary workflow. I don't really view it as two layers of window management.

Same for me, I use both tmux and neovim splits + tabs together and it works perfectly. I use tmux to launch a terminal next to the editor or tail logs while I use neovim splits and tabs to open separate documents.

EDIT: The flip side of the coin is that I have to memorize two sets of keybindings to navigate through the splits but In the end the distinction between terminal and editor splits is worth it at least for my use case.

vv9k avatar Jun 21 '21 16:06 vv9k

So consensus so far appears to be in favor of keeping the built-in splits. However, I'm strongly in favor of the ability to share buffers across windows.

CBenoit avatar Jun 21 '21 17:06 CBenoit

For me an additional advantage of being able to spawn several windows for the same editing session is to be able to use multiple monitors easily. Back when I used Emacs it was my main use for frames, for instance in order to have a compilation buffer on my secondary monitor and not clutter my main editing window.

Vim is really, really bad at this window management stuff IMO, so it's really not an example worth following (it still baffles me how anybody thought that Vim tabs were a good idea in their current implementation for instance).

simias avatar Jun 23 '21 12:06 simias

I would like to see some sort of a client/server session one day in the future. The editing model is built on operational transforms, so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

All this is of course a whole lot of work.

archseer avatar Jun 23 '21 13:06 archseer

two user clients

I wish the two clients can have different configurations but that is even harder.

pickfire avatar Jun 23 '21 13:06 pickfire

two user clients

I wish the two clients can have different configurations but that is even harder.

Actually it would probably be the default behavior. We would work with transactions, not configurations.

I would like to see some sort of a client/server session one day in the future. The editing model is built on operational transforms, so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

All this is of course a whole lot of work.

Not for tomorrow but very happy to hear that. I was very excited when I saw CoreMirror 6 collab API.

CBenoit avatar Jun 23 '21 15:06 CBenoit

Are transactions like CRDTs, or OTs?

kirawi avatar Jun 23 '21 17:06 kirawi

I like that Kakoune works very well with the Kitty terminal because of this. Kakoune itself can create a new Kitty split, then launch some process (like Ranger or Broot or a Git porcelain) inside that new window automatically, using the same mechanism as making another window for code. It makes the workflow feel very cohesive imo.

I don't mind having Emacs manage splits because Emacs has some Lisp package equivalent to every terminal app I'd ever use, but it seems really impossible imho for anything other than Emacs to achieve that.

Cons-Cat avatar Jun 27 '21 16:06 Cons-Cat

Perhaps this is something that would require an RFC?

kirawi avatar Jul 11 '21 17:07 kirawi

For me an additional advantage of being able to spawn several windows for the same editing session is to be able to use multiple monitors easily.

That makes a lot of sense. I also use multiple monitors, but I usually end up using my editor in only one of them. Maybe I would end up using the client/server model more than I think.

I assume you could have multiple different server sessions going at once, though, right...? I often find myself with multiple logically distinct editor sessions open (e.g. one working on a project, another on an unrelated file for an unrelated task that came up, and even occasionally yet another with a second unrelated project opened up), and I wouldn't want them accidentally cross-interacting due to being forced into the same server session.

cessen avatar Jul 12 '21 15:07 cessen

I assume you could have multiple different server sessions going at once, though, right...? I often find myself with multiple logically distinct editor sessions open (e.g. one working on a project, another on an unrelated file for an unrelated task that came up, and even occasionally yet another with a second unrelated project opened up), and I wouldn't want them accidentally cross-interacting due to being forced into the same server session.

With kakoune I open a new window on the same editor session by typing the new command in the editor (working on a single big project and I need several views), and a distinct editor session by opening my file with kak in another terminal. My common workflow is one big project with several windows and various terminals for unrelated files and stuff like you mention, it all works quite well :slightly_smiling_face: (Also, since I'm using i3 manipulating these windows is quite easy and convenient using only the keyboard)

CBenoit avatar Jul 12 '21 17:07 CBenoit

With kakoune I open a new window on the same editor session by typing the new command in the editor (working on a single big project and I need several views), and a distinct editor session by opening my file with kak in another terminal.

Okay, awesome. So it defaults to a new session, but you can opt-in to an existing session. That sounds good.

Ultimately, I think I'll still prefer having the application handle splits. Mainly because it's something that gets carried with the application, so I can always depend on its behavior regardless of environment. But you never know for sure without trying. :-)

(Also, since I'm using i3 manipulating these windows is quite easy and convenient using only the keyboard)

(I'm using AwesomeWM, so it's even easier for me. ;-) )

cessen avatar Jul 15 '21 15:07 cessen

With the renewed discussion on the Matrix channel about a client/server architecture, I've been thinking more about this. And I'm a bit concerned that the discussion over there is maybe heading in an "ultimate architecture that handles every use case" direction, which is something I've rarely seen work out in practice, and usually ends up with over-engineered solutions that collapse under their own weight.

So I'd like to suggest that for now we focus on just solving the "single user running sessions locally" use-case, and tackle other use-cases in the future once we have that implemented and really working well.

That may require re-working the architecture to some extent down the line to accommodate other use-cases. But I think it will also let us actually get something working in a reasonable time-frame, and furthermore will give us valuable experience that may inform how we want to handle those other use-cases.

Somewhat relatedly:

[...] so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

This is a very cool idea, and I like it. But I don't know if an editor-specific solution is really what we want here. For example, if I'm working at a company and want to pair-program with a coworker, I'm unlikely to convince them to use Helix for that purpose.

To give something like this any hope of being useful in practice, I suspect it would need to involve some kind of protocol that any editor can implement. And I'm not sure that's something worth tackling from within the Helix project. (Although experiments are always great, of course!)

cessen avatar Aug 16 '21 22:08 cessen

I'd like to point out that we can use https://github.com/zellij-org/zellij for this

VarLad avatar Aug 27 '21 23:08 VarLad

I thought how to solve this, and I came up with making an universal editor backend that would work with many frontends, and editors such as helix maybe could use that under the hood. I explained my vision at gbaranski/eb. What do you think?

gbaranski avatar Oct 05 '21 17:10 gbaranski

It seems to me that it may be helpful to draw upon the LSP and DAP protocols for inspiration here. They're both very mature and capable protocols that address significantly complex and variable aspects of the target abstraction.

kirawi avatar Oct 09 '21 01:10 kirawi

(I had to search for the DAP and LSP protocol pages, so I've linked both for convenience.)

haarts avatar Oct 12 '21 07:10 haarts

+1. I am excited for this project and I would adopt this 100% if it was built around a client-server architecture like a few other editors have been doing, the most notable being kakoune. My personal argument for this is that there are already a bunch of different window layering programs that do what they do, and window management from inside the text editor isn't necessary. I think this approach is exactly the approach of not attempting to "handle every use case", and the most workflow agnostic.

pantosaur avatar Dec 14 '21 20:12 pantosaur

Another benefit we would get from this client/server architecture would be the ability to share language-servers. Right now, each Helix instance will spawn its own language server. For instance, I sometimes use different Helix instances to simulate "tabs". Coming from Vim, I used tabs mostly as a context switcher. One tab would have buffers and splits related to a specific portion of the project, while another tab would be a different portion of the project. In Helix I spawn different instances to accomplish the same thing. Unfortunately, this also means that another language server spawns all in vein. A client/server architecture would solve this nicely. 🙂

hovsater avatar Jan 12 '22 09:01 hovsater

While we're at it, I would love it if the server/client design could also be used like EmacsClient I mentioned in #2177 with one mode where "hx-client" would just tell the running Helix instance to open the file and then return immediately, and a "wait" mode where hx-client would wait until the file is closed in Helix (useful for editing git commit messages in an already running editor instance).

johalun avatar Apr 19 '22 16:04 johalun

I'm very much in favor of a client-server architecture because it fits my workflow. My window manager is good at managing windows, my terminal multiplexer is good at managing the terminal space, and my editor should be good at editing files. Usually, I have a single terminal split into multiple panes (and tabs). One for version control, multiple for editing, and at least one shell. I switch/scale/move them via my terminal multiplexer. With helix, I keep a single pane for editing and have to use different shortcuts inside my terminal depending on whether helix runs inside the pane. This is mildly annoying.

robem avatar Sep 13 '22 15:09 robem

i need client-server. one of the main reasons i will be using emacs for the following years. but also make sure we can use this remotely smoothly; emacs client-server is not suitable for remote connections which is a pain

yuuyins avatar Sep 29 '22 16:09 yuuyins

How could this be handled in a cross-platform way? Windows doesn't support daemon processes like unix.

kirawi avatar Oct 19 '22 20:10 kirawi

We could do it in a peer 2 peer fashion, e.g. using libp2p, this could even allow connecting into a browser session. Which in turn could also enable collaborative editing over the network as well if done properly (I'm thinking of CRDTs)

Philipp-M avatar Oct 19 '22 21:10 Philipp-M

We could do it in a peer 2 peer fashion, e.g. using libp2p, this could even allow connecting into a browser session. Which in turn could also enable collaborative editing over the network as well if done properly (I'm thinking of CRDTs)

Yeah I have branches on both of these: Used diamond-types for CRDTs, and libp2p for a direct collaborative session. I think this is a bit orthogonal to this issue though.

archseer avatar Oct 20 '22 03:10 archseer

Oh really, cool :)

Do you have the branches public somewhere? I haven't found them yet.

CRDTs are certainly something separate. (lib)p2p though I think can indeed help abstract the networking/communication/discovering layer away.

For example (rough idea):

  1. At the startup of helix discover other clients locally
  2. Check which documents are open in each of these instances map each instance as master of these documents
  3. If a client that is not master of a document, wants to open that document, it connects to/registers at the master of that document
  4. Every time an edit occurs, it is transferred to the master, which does the actual edit
  5. When the master gets closed, it could transfer the document state to the first connected slave. I guess this will get tricky, because we would need to make the state Serializable (related #401)

Edit: Thinking about it, this approach would make handling with language servers quite difficult though, it may need some namespacing for the language server (roots?)

Philipp-M avatar Oct 20 '22 17:10 Philipp-M