Extending god-mode system wide outside of Emacs

[GeekyPeas]

I love god-mode so much that I would actually want to extend it system wide outside of Emacs. I want to do most of the cool thing that god-mode does inside Emacs, outside of Emacs!

I have an idea about how to go about this: We could hack Plover and replace its translation engine with god-mode running on an Emacs daemon. But at this moment I don't have all the required skill set to pull this off! A project like this will need community involvement and contributions (and I will surely try to do my best to contribute!)

(Edit: Segway: What is plover? Plover project from the Open Steno Project is an open-source stenography engine, that can translate key-chords on an N-key rollover-keyboard into English words via a predefined dictionary. Stenography uses key-chords and phonetics for input. For example, OESHN becomes Ocean and FASHN becomes Fashion. See this for more examples. I have no stenography background, but from what I have found online, Art of Chording is considered a good standard reference on stenography.)

There has to be some sensible changes that have to be made to god-mode outside of Emacs. For example, inside Emacs god-mode would translate 5n to C-u 5 C-n. But outside Emacs 5n is needs to be further translated to C-n C-n C-n C-n C-n. I don't know at this moment what should be all those sensible changes. As a community we can figure such things out and refine them with trial and error and natural evolution over time.

So I intend to use this github issue, as an advertisement/ brainstorming place for the above project. I would have liked to put something like this on a discussion page of god-mode. But as suggested here I am putting this issue here.

Plover may not be the best route forward. There are things called Input Method Engine/ Input Method Framework in Linux. So I don't know if Input Method Engine will be a better place to start at.

I asked a related question on stackoverflow. That question is more open-ended and doesn't insist on implementing god-mode system-wide as the only solution. But for my personal preferences, I would love god-mode system-wide! And I hope this github issue will reach out to others who may also love god-mode so much that they just want it everywhere!

p.s.: My rambles/ opinions about other (somewhat) related ideas:

1. Vim-keybindings: have such cult following that they are popular on many interfaces outside of vim. I feel God-mode deserves that kind of a place! Unlike vim-keybindings, what I like about god-mode is that it doesn't replace your already familiar keybindings with something completely different. And further, vim-keybindings have to be remapped if one wishes to switch to a Colemak or a Dvorak layout, where as God-mode keybindings can remain unaltered! In that sense, I feel god-mode has more universal appeal, and we need to ascend God-mode to its rightful place!
2. custom keyboards: are really cool. People have done some really nice work with them. Project ergogen definitely deserves accolades/ special mention. People are able to have modal layers on their keyboards at a hardware level! But sometimes you may have to work on a "standard keyboard" at work or at your friend's place. It will be nice to have a software ecosystem that can quickly enable god-mode system wide so that you can get your job done and then revert back to defaults for others.

[GeekyPeas]

I posted on the Doom Emacs Server the above issue as an advertisement to catch attention of other enthusiasts! Here is the posting.

[QiangF]

What is plover? There are several key remappers that might do what you want. Such as xremap, xkeysnail, hawck. They are all on github.

[GeekyPeas]

@QiangF : Thanks for telling me about xremap, xkeysnail and hawck. Until now I only knew about xmodmap.

Quick questions, in case you are already familiar with some of the above:

1. Do the above support long key sequences? For example can it translate xp V to C-x C-p V?
2. Are the translations programmable? By that I mean, will it know that 3n should become C-n C-n C-n and 5n should become C-n C-n C-n C-n C-n without having to enumerate them all in config declaration but just programmatically defining them once!

And apologies for not explaining what is Plover. I have edited the main post above and added a short description on Plover. Basically Plover is a steno engine that can translate key chords into words. It supports modifiers like Ctrl, Alt etc I believe.

[GeekyPeas]

I wonder if it will be a good idea to not take on the burden of defining god-mode outside of Emacs! Instead can one consult an Emacs daemon running god-mode for what the translation should be? Basically use god-mode running in Emacs as the config file/ dictionary/ back-end engine for the above remapping software.

Pros: If god-mode changes (either at source or at user), one wouldn't have to do extra work to keep these remappers consistent with the changes.

Cons:

1. Do god-mode keybindings even make sense outside of Emacs?! The Emacs key-binding convention is what makes god-mode so useful. So one may anyway have to modify the default behavior of god-mode when extending it outside of Emacs.
2. If one wants to make god-mode more universally available, then the fewer the dependencies the better! So making god-mode depend on Emacs to work may be a bit too restrictive.

Personally: I like the idea of using Emacs as a config file replacement/ back-end engine for the above remappers!

[darth10]

Glad to hear you enjoy god-mode so much @GeekyPeas!

I wonder if it will be a good idea to not take on the burden of defining god-mode outside of Emacs!

god-mode translates keys pressed into commands and invokes the commands (string -> command -> effect). Thus, god-mode is coupled with Emacs commands. This is an implementation detail, but we could change this to have an intermediary function return key bindings instead without too much hassle (string -> string).

Coming to the idea of porting god-mode to any other key mapper software, I feel that's a viable approach too. For example Evil (vi bindings), which runs in Emacs, doesn't use a vi/vim child process. In fact, no plugin for any editors (like VSCode) emulates vi bindings by running a vi/vim child process. There's a tangible performance cost that comes with inter-process communication that's not ideal for the use case of text editors and OS-level key mappers.

The semantics of god-mode for reducing modifier keys pressed won't really change, but the customisation options available to Emacs users will improve, without breaking backward compatibility of course. If the semantics do need to change, we would create a new Emacs package instead of modifying god-mode. So, porting god-mode semantics to any other tool isn't a bad idea.

[darth10]

Do god-mode keybindings even make sense outside of Emacs?!

That's a good question, but I think the answer is yes. Lots of tools use modifier keys, especially C-, used for a sequence of keys entered.

[GeekyPeas]

@darth10: Thanks for the many helpful inputs! It is very encouraging to hear!

There's a tangible performance cost that comes with inter-process communication that's not ideal for the use case of text editors and OS-level key mappers.

I hear what you say, and I am mostly convinced with the argument you are making. In particular, I would consider the following to be very compelling evidence for your argument:

For example Evil (vi bindings), which runs in Emacs, doesn't use a vi/vim child process. In fact, no plugin for any editors (like VSCode) emulates vi bindings by running a vi/vim child process.

However I am not entirely convinced. Here is why:

1. xkeysnail is keyboard remapping tool for X environment and it is written in Python! Python as we know has a reputation for being a slow language. So people do lazily get away with/ tolerate such inefficiencies!

2. EXWM is complete X Window Manager that runs on Emacs. So this is evidence that Emacs handles fine the OS-level activities. And yes there are people who complain about how Emacs is single-threaded and so EXWM ought to be a terrible idea. And yet there are people who love EXWM.

disclaimer: I don't understand inter-process communication and its overheads very deeply. So in the above two examples, I may have overlooked something important. Please let me know!

So my takeaways:

1. We should not insist on having Emacs as a back-end.
2. We also should not very aggressively discourage on having Emacs as a back-end.

(That is, let people decide what they want and remain Emacs agnostic/ neutral here).

And thanks for explicitly distinguishing god-mode from its semantics. That has fostered a lot of clarity for me!

[darth10]

Both xkeysnail and EXWM are good tools written in languages not close to the hardware. I'm not well versed with xkeysnail, but I know EXWM well enough. EDIT: I had a look at xkeysnail, and it also uses Xorg indirectly through evdev.

I know that EXWM uses XELB, which uses the X11 protocol to communicate with Xorg. That's not exactly interprocess communication with a child process, but sure it's some form of interprocess communication. Xorg is implemented in C (maybe some C++) AFAIK. So the daemon process actually needs to have as little overhead as possible, which won't always be the case with Emacs. Maybe Emacs with native compilation is a bit better, but that's still far from ideal performance due to having a Lisp interpreter relaying calls to a natively compiled package.

To take this argument even further, vi is also implemented in C, but it's never used as a child process to emulate vi bindings.

I'm not discouraging the use of Emacs as a back-end, but there are certain aspects of that approach which need consideration.

[darth10]

Also, just FYI, I've used EXWM in the past, but I switched to stumpwm and never looked back. The single-threaded nature of Emacs doesn't have a great appeal to me personally as a reliable window manager. IMHO EXWM is great example of what Emacs is capable of, but it's only viable with a future version of Emacs that has proper support for multithreading.

[darth10]

I had a look at xkeysnail, and it also uses Xorg indirectly through evdev.

Maybe using evdev, through packages like python-evdev, and uinput is a good approach.

Modifier key translations done by god-mode can easily be implemented in a dozen lines of Python or any other language. Universal arguments are a feature native to Emacs though. The concrete semantics of universal arguments could also be ported to another tool/language, but that would be non-trivial.

[QiangF]

EXWM has bad input method support for qt5 applications. https://github.com/ch11ng/exim/issues/6

If you care of speed, you should try its successor https://github.com/joshgoebel/keyszer

[TheGitMux]

keyd is appropriate for remapping keys.

[GeekyPeas]

Update: As I have been a bit busy with some personal work, I am unable make time for this right now. I plan to commence work on this in the first quarter of next year. (I feel this update may not belong here on this git-hub issue, but I don't know of a better place as well).

Many thanks to everyone for your suggestions!

The main objective of this update is to announce that - this project is not dead for me personally.

And of course, for anyone else, I don't intend to own or control this project. So please feel free to contribute, while I am away, if you are inclined to.