Support emacs with cursorless

[ParetoOptimalDev]

trafficstars

• [ ] abo-abo/avy#341
• [ ] tree-sitter support in emacs (option 1)
• [ ] #435
• [ ] RPC is needed? Current method good enough? (persistent TCP socket example )
• [ ] create issue to support in LSP-mode if necessary

[pokey]

Thanks for opening this one! I'll keep you posted on progress re #435. Re sending commands to emacs, you might touch base with @rntz; iirc he's got some kind of communication layer working

[pokey]

The other person you might touch base with re sending commands to emacs is @jcaw. I believe they may have a fairly sophisticated emacs rpc setup that might be worth checking out if it's open source

[jcaw]

Spoke to @ParetoOptimalDev about this on Slack, but I'll summarise here for posterity. I currently have two public projects for Emacs RPC:

• Porthole allows you to send commands via transient HTTP requests. It's designed to be plug-and-play, and has a prewritten Python client. I used to use Porthole as my RPC layer. Porthole is on Melpa. As I understand it still works fine, but I don't explicitly maintain it. Do report any issues and I'll take a look.
• Voicemacs uses a persistent TCP connection with a custom protocol. It's the setup I currently use (it's faster and more flexible than transient HTTP connections) but it's not designed for end-users. I publish the code as a reference but I wouldn't recommend using it directly.

Another option is to send Elisp to a running Emacs server via the emacsclient command itself. I know a few people have taken that approach in the past.

[adabru]

I'm currently creating a kind of extended onscreen keyboard and need to decide on a protocol to communicate with talon.

Thanks pokey and team for contributing your protocol to the community :) ! For my use case I'm concerned about latency introduced by the file-based ipc. Furthermore I need a more bidirectional communication to send commands and set and get tags. From https://github.com/knausj85/knausj_talon/tree/main/apps/vscode/command_client I'm inferring that only talon initiates communication in cursorless.

The talon wiki "recommends" bash -c "echo 'actions.speech.toggle()' | ~/.talon/bin/repl" to send commands to talon.

https://github.com/jcaw/talon_config/blob/master/emacs/utils/voicemacs.py seems like a fitting base for my use case. It provides bidirectional and also threaded communication (less latency in my experience). Any chance pokey that you could make use of it for cursorless? Or do you think it's not worth trying to mostly unify the RPC/IPC protocol yet?

[phillco]

@rntz and @dyamito have an implementation going here :) https://github.com/cursorless-everywhere/emacs-cursorless

[pokey]

@adabru yes the lack of bidirectionality is definitely a limitation. I have a draft PR to add support for bidirectional communication, but it needs to be reworked a bit

Re latency, have you had issues with latency when using the file-based RPC? In my experience there is no perceptible latency

[adabru]

Thanks @phillco for showing me that!

@pokey: No, it's based on suspicion 🙈 If it's relevant enough I can try to measure it lateron 👍 My criteria would be around 10ms, but I would call anything >1ms slow.

Actually I wrongly assumed the current vscode-side server was based on file watching. Instead it is based on key listening from vscode. The linked talon-side code in the PR is based on file watching though. After some code digging I think now that cursorless' middleterm plan is already to use unix sockets (sidecar version) for the requests and just send the payload via files.

[adabru]

Back to unification: I compare following protocol implementations:

voicemacs talon client voicemacs emacs server cursorless emacs client cursorless talon client cursorless vscode server delegation cursorless vscode server next cursorless talon server cursorless jetbrains server VisualStudio VisualStudio server delegation VisualStudio talon client

The current VisualStudio and vscode server+client use the same data exchange, ie. trigger via key, payload via json-rpc in files with a specified path. voicemacs seems to use json-rpc and a custom json format via tcp socket at a specified port. The sidecar and jetbrains versions use yet another protocol. So I want to find out whether these use cases can kinda be integrated into talon's current command server and command client. As a result it shall fit all above mentioned usecases, my usecase, and hopefully that of more future users than the current one.

Implementation-wise the protocol would be independent from talon. I looked into several third party cross-platform, cross-language solutions (zeromq, yami4, dbus, json-rpc) but none of them seemed 100% fitting. D-Bus would've been nice, it also resembles file based rpc in some way, but it has a non-trivial cross-platform distribution path.

Following is an example of how a bus-style result could look like when it's used:

pseudo code

bus = SessionBus("path/to/socket")
// alternatively with file watching
bus = SessionBus("path/to/command_dir")


// expose local object to bus
bus.register("vscode.command_server", local_object)


// consume remote object
try:
	proxy = bus.get("vscode.command_server")

	// request and response
	rpc_request = proxy.call("run_command", "user.snippet_insert", "loop")
	await rpc_request.result()

	// publish and subscribe
	local_object.tag_changed_signal.notify("sleep", true)

	proxy.subscribe("tag_changed_signal", local_callback)

catch Timeout: ...
catch Malformed: ...
catch NotAvailable: ...
catch NoServer: ...
catch Authentication: ...

// bus functions
print await bus.listObjects()
bus.subscribe("object_changed", local_callback)
print await proxy.inspect_interface()

Based on the voicemacs emacs server I'd expect the server implementation to be around 2k-3k lines of code while a pure client implementation could be around 500 loc. The user facing api is the same. At least one client must implement a server. Talon's client should be one of them. Connecting multiple applications to the same bus should be supported.

[adabru]

The POC server implementation was only ~700loc. You can check it out with https://github.com/adabru/eyeput/blob/main/session_bus_test.py .

As aegis mentioned on slack he will add rpc into 0.4 and both phillco and rntz implemented the file system version, I will no longer pursue a protocol update.

[jcaw]

It may be worth noting, a lot of the Voicemacs networking code is there to negotiate with Emacs' archaic network processing - direct Python to Python should be easier to implement. Voicemacs is also a little janky - I threw it together as a low-latency solution for persistent RPC to Emacs, but I'm holding off rewriting it until Talon has a defined RPC system. Emacs is particularly bad for latency because it's single-threaded (and from the stone age).

I think my old Emacs networking projects are also overengineered. They're more complex than they need to be.

[jaresty]

I'd really love to be able to use cursorless in my emacs config. I'm curious - how has this been going?

[pokey]

We recently decided that the approach to integrating Cursorless into other IDEs need not be a one-size-fits-all approach. There are three possible approaches:

• Bundling Cursorless into a language server, and letting the IDE implement a language client that talks to the server using LSP
• Letting the IDE directly import the Cursorless engine as a Javascript NPM package
• Bundle Cursorless into a wasm and let the IDE run the Cursorless wasm

In all cases, the Cursorless core maintainers will be responsible for the part of the Cursorless inside the “interface boundary”, ie the Cursorless LSP server, the Cursorless NPM package, or the Cursorless wasm. The IDE-specific code that interacts with the Cursorless interface will be maintained by someone who is responsible specifically for the integration with the given IDE.

Given that we are considering three possible ways to integrate, we have decided not to go with a waterfall approach where we invest in picking one approach, implementing it, then letting IDE-specific contributor build the client.

Instead, we’d like to work closely with a couple IDE client maintainers so that we can figure out which approach makes the most sense for their IDE. If you’re interested in being one of those IDE client maintainers, and have some capacity to hack in the next couple months; let us know, and then drop into a meet-up so we can get started!

[jaresty]

Thank you for the update! That approach makes sense to me. I don't think my emacs configuration skills are really up to the task though I appreciate the offer.