Feature request: workspace/executeCommand (how to implement?)

[spchamp]

For the Eglot client with Emacs 29 on FreeBSD, to communicate with solargraph via a workspace/executeCommand LSP message, I've defined the following function:

(defun eglot-send-eval (str)
  (interactive "Meval: ")
  (eglot-execute-command (eglot-current-server) "eval" (vector str)))

Albeit, there would probably need to be a server-side handler for an 'eval' command, in order for this to be of any particular use. I'm not entirely certain if I have the argument syntax right for that protocol call in eglot. The function does not succeed - it results in a jsonrpc-error in Emacs. Perhaps it's of some use for understanding how the LSP protocol is implemented in client and server.

After I've called this function as an Emacs Lisp command in an Emacs buffer with an active solargraph server, such as to send a string "true" as an expression for this ostensible 'eval' command - albeit while there may not presently be any server-side code available as to receive the expression, evaluate the expression, and then provide any representation of a return value to eglot - I see the following output in the eglot events buffer:

[client-request] (id:2) Mon Mar 14 22:05:56 2022:                                                                                                                       
(:jsonrpc "2.0" :id 2 :method "workspace/executeCommand" :params                                                                                                        
          (:command "eval" :arguments                                                                                                                                   
                    ["true"]))                                                                                                                                          
[server-reply] (id:2) ERROR Mon Mar 14 22:05:56 2022:                                                                                                                   
(:jsonrpc "2.0" :id 2 :error                                                                                                                                            
          (:code -32601 :message "Method not found: workspace/executeCommand"))     

Perhaps this message/method might be documented for LSP 3.16

Simply, I was hoping to try to figure out how to implement something like an inf-ruby or IRB read/eval/print loop over solargraph and eglot. Looking at the protocol documentation, I believe that this workspace/executeCommand interface might be the closest thing to an eval form - or perhaps it could at least carry an eval request, in a sense like an input line in IRB or Pry.

If the workspace/executeCommand interface may need an implementation, I'd be interested in contributing code for this. If it may not be too broad of a question, could there be any advice about where to look in the source code?

[spchamp]

If it may be helpful to share some thoughts about how an eval cmd could be applied, hopefully this could be a useful notes page at some near time:

• Unlike in inf-ruby or irb, the implementation might entail some form of asynchronous eval.
• It may not be guaranteed to be thread-safe for all calls to 'eval.' Whether with a host mutex and any relevant condition variables, or anything from concurrent-ruby hopefully the eval calls could be at least be partially guarded against possible side effects from concurrent eval.
• The eval command could be extended beyond the simple string argument signature, to accept a required or optional second argument representing an eval-tag (string) such that would be produced by the client and could be returned with any complete or partial-results response, as to indicate which eval request had originated the response.
• In emacs, it might not be possible to use comint here. comint might be suitable for synchronous forms of eval, albeit possibly agumented with job control as under sh, bash, zsh, csh, ksh, other
• IRB offers a form a workspace object, perhaps in a sense like an environment or an eval context. If the implementing Ruby has an IRB avaialble, maybe some of that code could be reused here, on the server side? inf-ruby itself might provide an example of how to access a binding object for eval, from within an IRB workspace - the binding object might be the main thing to focus on, for the eval?
• Error handling? If the eval string results in an error, hopefully that could be handled atomically. Maybe it could be possible to hook an interface for interactive debugging onto this.
• Multi-line evaluation? I've not looked really closely at the IRB source code, to see if it may be provide a public API onto how IRB provides an interactive 'eval'. IRB provides some configuration options affecting how the interactive prompt will be presented during multi-line eval. Maybe it could be useful to look at, for implementing an eval command in solargraph?
• If there would be a distinct "eval response" message, maybe it could include a byte as to indicate whether the server assumes that the input will be continued in some subsequent request. The client could then reuse the original eval tag for any continued input / continued eval
• Will the server need to hold onto any references for return values? or simply produce a string indicating what the return value was? Maybe that could be clarified at some point during a protocol design - something like a ring buffer, for eval results and exceptions?
• There must be a way to implement unit tests for this? mockups, even, or an LSP client API in Ruby?

I'll take a look at the code again.Though I'd begun working on a fork of inf-ruby, the development had got fairly stuck at the question of how to implement an asynchronous eval there. I think solargraph and eglot could really add a lot of useful features for interactive coding with Ruby, even if it may need some extensions to the language server protocol

[spchamp]

To the question of where to look in trying to extend the language server protocol support: In a gem installation for solargraph, I see a file solargraph-0.44.3/lib/solargraph/language_server/message.rb defining a module Solargraph::LanguageServer::Message. In the module's definition, it seems that the module's register method is called as to define some protocol forms for the language server protocol. e.g in excerpt

      register 'initialize',                          Initialize                                                                                                        
      register 'initialized',                         Initialized                                                                                                       
      register 'textDocument/completion',             TextDocument::Completion                                                                                          
      register 'completionItem/resolve',              CompletionItem::Resolve 

Perhaps this is where the extension methods for solagraph are hooked in. Maybe it could be more effective to define an $/extensionMethod than trying to fit an 'eval' protocol in over workspace/executeCommand, but perhaps the protocol forms for the latter could help to serve as any guide here.

Focusing on the message form:

    register 'textDocument/documentSymbol',         TextDocument::DocumentSymbol

...Solargraph indicates that this is a reference to Solargraph::LanguageServer::Message::TextDocument::DocumentSymbol which is a class.

Finding a definition of this class in solargraph-0.44.3/lib/solargraph/language_server/message/workspace/workspace_symbol.rb I see that the class defines a process method. Another example of this method's implementation would be available in solargraph-0.44.3/lib/solargraph/language_server/message/text_document/definition.rb. This process method might be a useful place to start, for contributing an implementation of workspace/executeCommand?

Perhaps the semantics of the workspace/executeCommand message might seem to differ generally to the file-oriented nature of most other protocol messages under the workspace qualifier.

imo the LSP protocol itself may not seem to have been designed for supporting interactive eval. Perhaps it's more oriented towards statically compiled languages and desktop IDEs. I think it's possible to implement an 'eval' protocol with these messsage forms, however.

I'll take a look at the definition of the Workspace messages, maybe there's a way to patch in an implementation for a process method under a class providing an implementation for workspace/executeCommand

[spchamp]

When I was tinkgering with some of the Gems defined under the ruby-gnome codebase, once, along with reviewing the GNOME devehlp documentation that may accompany the GTK APIs, I'd noticed that there's a dbus interface available in GTK. Towards making use of an eval protocol in solargraph, perhaps it may be feasible to define a glib/dbus service for solargraph. However nontrivial that may be at the point of implementation, perhaps it could be addressed more generally for integrating any LSP servers in a glib environment with some LSP client interface, broadly in a generalization of Eglot onto glib - ostensibly, via Ruby though. Perhaps it could be approached as to define something like an IDE service onto glib.

Continuing with an ostensible design for workspace command support in solargraph: If the LSP protocol does not define any specific message syntax for the 'command' and optional 'arguments' features of the ExecuteCommandParams message interface, and does not define any protocol message for the server to indicate what commands or command syntaxes it may support ... the command string itself could be represented like with a qualified name e.g devo.example.com/eval or even devo.example.com/cd, devo.example.com/pwd, devo.example.com/sh, devo.example.com/supportedCommands etc.

For purpose of coordinating any server and client capabilities, maybe these commands could be described with an OpenAPI syntax (YAML or JSON) if any qualified names for any server-supported commands may not be sufficient. Perhaps that may not be a lot of useful for any connecting clients, however - e.g traversing each command description for some sort of a predicate test as to whether it matches what the client supports for any similarly named command string. Alternately, the server-originated client/registerCapability message form could be of use here.

Towards something like a semantics for reporting return values from the server, perhaps each workspace command request could include something like the ProgressToken string from the $/progress message support in the LSP protocol. Albeit, I'm not certain if this is even supported in Eglot, presently. Maybe there is a reference implementation of this $/progress part of the protocol, in an LSP client application, somewhere. Alternately, maybe this could be added with some contrib code for Eglot, if there is a reference implementation of $/progress messaging on the server side.

Whether or not as an implementation of the $/progress aspects of the protocol, ostensibly the client presenting any workspace/executeCommand message could provide a command token unique to each initial command, within the same message - whether as a value in the optional command arguments field or as some sideband value like a ProgressToken. The server might then reuse that command token as a ProgressToken in any responses, to allow the client to match each response to each initial, asynchronous command.

On the server side, any synchronous command implementations (e.g cd) could return a progress report of 100% indicating success, or some response if the command has failed. Any asynchronous command implementation on the server (e.g a command to build some package from the current workspace) it could provide incremental progress reports with some polling to monitor progress on the server side. A progress percentage of zero could be used when the server cannot estimate a percentage complete for any asynchronous-on-the-server command, until the comand has completed.

Possibly the details of this could be worked out with some contrib code for eglot, to have the workspace/executeCommand implementation worked for client and server implementations, before implementing a glib/dbus service for any connection dispatching in LSP server/client communications

[castwide]

I appreciate the thought you've put into this feature, but unfortunately, supporting eval in Solargraph seems unfeasible. Solargraph works mostly through static analysis and executes very little workspace code by design. Evaluating code on the fly seems better suited to a debugger.