How to debug language server

[MilesMcBain]

Hi

How can users of the language server debug it?

I have a valid R file which seems to reliably cause a textDocument/documentSymbol to time out without a response.

I would like to try to debug a running language server, but I am struggling a bit to figure out the workflow.

I started to use tests/testthat/test-symbol.R as a template and I was able to observe this in the log:

[2021-02-16 22:43:16.866] did save: {
  "uri": "file:///home/miles/code/r/blogdown_test/content/post/test.R"
}
[2021-02-16 22:43:17.089] parse_callback called: {
  "uri": "file:///home/miles/code/r/blogdown_test/content/post/test.R",
  "version": null
}
[2021-02-16 22:43:17.641] resolve_callback called: {
  "uri": "file:///home/miles/code/r/blogdown_test/content/post/test.R",
  "version": null
}
[2021-02-16 22:43:18.153] received:  Content-Length: 159
[2021-02-16 22:43:18.164] received payload.
[2021-02-16 22:43:18.165] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:18.165] document definitions found:  0
[2021-02-16 22:43:18.166] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:18.572] received:  Content-Length: 159
[2021-02-16 22:43:18.584] received payload.
[2021-02-16 22:43:18.585] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:18.586] document definitions found:  0
[2021-02-16 22:43:18.588] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:18.896] received:  Content-Length: 159
[2021-02-16 22:43:18.907] received payload.
[2021-02-16 22:43:18.907] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:18.907] document definitions found:  0
[2021-02-16 22:43:18.908] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:19.216] received:  Content-Length: 159
[2021-02-16 22:43:19.228] received payload.
[2021-02-16 22:43:19.229] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:19.230] document definitions found:  0
[2021-02-16 22:43:19.233] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:19.645] received:  Content-Length: 159
[2021-02-16 22:43:19.657] received payload.
[2021-02-16 22:43:19.658] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:19.659] document definitions found:  0
[2021-02-16 22:43:19.660] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:20.071] received:  Content-Length: 159
[2021-02-16 22:43:20.083] received payload.
[2021-02-16 22:43:20.084] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:20.086] document definitions found:  0
[2021-02-16 22:43:20.088] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:20.396] received:  Content-Length: 159
[2021-02-16 22:43:20.408] received payload.
[2021-02-16 22:43:20.409] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:20.410] document definitions found:  0
[2021-02-16 22:43:20.413] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:20.822] received:  Content-Length: 159
[2021-02-16 22:43:20.835] received payload.
[2021-02-16 22:43:20.836] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:20.837] document definitions found:  0
[2021-02-16 22:43:20.839] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:21.252] received:  Content-Length: 160
[2021-02-16 22:43:21.264] received payload.
[2021-02-16 22:43:21.265] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:21.266] document definitions found:  0
[2021-02-16 22:43:21.269] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:21.681] received:  Content-Length: 160
[2021-02-16 22:43:21.693] received payload.
[2021-02-16 22:43:21.693] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:21.694] document definitions found:  0
[2021-02-16 22:43:21.696] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:22.107] received:  Content-Length: 160
[2021-02-16 22:43:22.119] received payload.
[2021-02-16 22:43:22.120] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:22.121] document definitions found:  0
[2021-02-16 22:43:22.122] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:22.534] received:  Content-Length: 160
[2021-02-16 22:43:22.546] received payload.
[2021-02-16 22:43:22.547] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:22.549] document definitions found:  0
[2021-02-16 22:43:22.550] deliver:  ["Response", "Message", "R6"]
[2021-02-16 22:43:22.962] received:  Content-Length: 160
[2021-02-16 22:43:22.974] received payload.
[2021-02-16 22:43:22.975] handling request:  textDocument/documentSymbol
[2021-02-16 22:43:22.976] document definitions found:  0

loop continues...

So it seems there is some issue in the symbol parsing setup. I would like to stick a browser() call in document_symbol_reply() to try to figure it out, but then need to be running the languageserver in an interactive session. So I can't figure out how to start a server interactively that I can address e.g. in my interactive session I tried:

languageserver::run(debug = "~/repos/languageserver/logs/log.txt", port = 8888)
Error in socketConnection(host = host, port = port, open = "r+b") : 
  cannot open the connection
In addition: Warning message:
In socketConnection(host = host, port = port, open = "r+b") :
  localhost:8888 cannot be opened

Is this supposed to work? Looking into the code... wouldn't it need socketConnection(server = TRUE)?

Is there some way I can use the stdio mode and connect a language server client to a language server that isn't in a child process?

A bit lost. I am sure you have a good workflow for this. Please share!

[renkun-ken]

What do you get with the following:

file <- "path/to/your-file.R" 
env <- languageserver:::parse_document(file, readLines(file))
ls.str(env)

[MilesMcBain]

It seems that this part is working:

> ls.str(env)
definitions : List of 4
 $ foo:List of 3
 $ bar:List of 3
 $ baz:List of 3
 $ var:List of 3
documentation :  list()
formals :  list()
functs :  chr(0) 
nonfuncts :  chr [1:4] "foo" "bar" "baz" "var"
objects :  chr [1:4] "foo" "bar" "baz" "var"
packages :  chr "nycflights13"
signatures :  list()
xml_data :  chr "<?xml version=\"1.0\" encoding=\"UTF-8\"standalone=\"yes\" ?>\n<exprlist>\n<expr line1=\"2\" col1=\"1\" line2=\"| __truncated__
xml_doc :  NULL

But my question was really: Is there a workflow for interactively debugging the languageserver code?

[MilesMcBain]

The way I would have expected it to work would be something like:

1. In session 1, place browser() or use debugonce() on function
2. In session 1, run languageserver::run(port = 8888) interactively
   • this doesn't seem to work? bug?
3. In session 2, create a new client that connects to that server, e.g. client <- LanguageClient$new(port = 8888)
   • Doesn't seem like there's a way to point a client at a running server over TCP?
4. In session 2, call respond_symbol(client, ...)
5. In session 1, interactively debug the symbol response.

[MilesMcBain]

So I found that my problem was related to an old languageserver version. documentSymbol works with the latest one.

In the process I made a small fix to the server to allow it to actually run in TCP mode and made a TCP client class analogous to the existing languagclient that allows interactive debugging like I describe above using TCP. Any interest in a PR for this?

https://github.com/milesmcbain/languageserver/blob/tcp-client/R/tcplanguageclient.R

[randy3k]

@MilesMcBain what is the specific fix that you need for tcp?

[randy3k]

OIC. You want a client that uses tcp connection. I am not sure though if we should factor it in the existing ~~server~~ client or put it in a different class.

Anyway, it seems a great idea to use tcp correction for debugging. Stdio doesn't make it easy.

[MilesMcBain]

For me the fix for tcp was this: https://github.com/MilesMcBain/languageserver/blob/tcp-client/R/languageserver.R#L48

I can't bind the ip/port without server = TRUE

To me it made sense to make a separate client class, because it used the much the same interface as the existing client with different backend. The other option would be testing the mode inside each client method? Adds a bit of complexity to code?

[randy3k]

server = TRUE is needed to allow the client to listen for connections. It may be confusing, but the language client is actually the server of the tcp connection and the language server is on the other hand the client of the tcp connection.

[MilesMcBain]

😆 🤯 This is classic Microsoft.

Thankyou for that. I don't think that should affect what I have done too much. It should be simple to flip that around. When you're debugging you're in the driver's seat.

[hongooi73]

@MilesMcBain Microsoft is just channeling X Windows here.... 😅

[MilesMcBain]

I wrapped up this debugging workflow as a new package, because I needed it for another PR I am in the process of making. You can try it and see if it's useful: https://github.com/MilesMcBain/tcplspclient.

[randy3k]

Interesting, you could also generalize the idea and create a headless editor that allows arbitrary language server support.