Extending to nvim-dap?

[sebffischer]

Sorry for this probably stupid question, but in neovim there is the possibility to use nvim-dap for languages for which the debug adapter protocol was implemented. Is it possible to use the VSCode-R-Debugger also in this scenario (with some modifications)?

[ManuelHentschel]

Definitely not a stupid question! I had the same idea some time last year and actually spent some time working on a solution in https://github.com/ManuelHentschel/dapWrapper (the dap-server seemed to work with different clients, but was quite buggy). The main problem with the vscDebugger package is that there seems to be no way to control the program flow (stepping through code while debugging) from within R itself. Therefore, I am using a node-based wrapper, which controls the input to/output from an R process underneath.

Also, I am currently playing around a bit with Rcpp and boost.asio and hope that I'll be able to implement a dap-server that is able to run in the background of a user-managed R session, which might make it easier to attach to an already running R session, using different dap-clients.

[sebffischer]

Thank you for your efforts! I would love to contribute but I don't think I have any relevant expertise... Why is it not possible to control the program flow from within R itself? (Now this is probably a stupid suggestion but I am trying to understand the problem.

If you want tto step through the current problem from within a running R session couldn't you simply save the current R source-code file as a .txt file, e.g. as "~/R_temp.txt", and then do the following:

src = readLines("~/R_temp.txt")
src = paste0(src, collapse = "\n")
src = parse(text = src)

# step through the code: 
eval(src[1])
eval(src[2])

Again, I really appreciate your work :)

[sebffischer]

Stepping into functions might be trickier, but I guess it might be doable as well.

[ManuelHentschel]

If you want tto step through the current problem from within a running R session couldn't you simply save the current R source-code file as a .txt file, e.g. as "~/R_temp.txt", and then do the following: [...]

That's roughly how source and .vsc.debugSource work.

Stepping into functions might be trickier, but I guess it might be doable as well.

This is definitely the tricky part, if you figure out a way to do this, without needing to write to stdin, please let me know!

[sebffischer]

I might give it a go in the christmas holidays. Are there any insights that you have gained so far?

[sebffischer]

Especially, have you tried to manually execute the function calls?

Something along the lines of:

# standard function call 
f = function(x) {
  print(x)
  return(x)
}
f(1) 
#> [1] 1
#> [1] 1

## manually constructing a function call
func_env = new.env() 
delayedAssign("x", 1, eval.env = func_env) 
eval(expression(print(x)), envir = func_env)
#> [1] 1
eval(expression(return(x)), envir = func_env)
#> [1] 1

^{Created on 2021-12-10 by the reprex package (v2.0.1)}

[sebffischer]

I mean probably you have, but what were the problems that you ran into?

[sebffischer]

Ok so this was probably a quite naive suggestion, with all the call stack stuff this would probably have to be done in C? Did you also try to implement it there or only in R?

[ManuelHentschel]

I agree that the straightforward approach you mentioned would be problematic, since it would mess up the call stack and you would have to use a modified version of eval recursively, since otherwise you would not be able to handle breakpoints in nested function calls.

Also, I would like to be able to handle browser() statements in functions that were called directly by the user and the way that the browser seems to be implemented is quite reliant on the input from stdin. I had a look at the R source code at some point and had the impression that it checks for the inputs n, c, etc. before even parsing the entire line, which would be hard to imitate from inside an R function.

Though until now I have not used C for anything but some simple helper functions, so maybe there is some potential for improvements there!

[randy3k]

I think it should be relatively easier in radian to control the flow because we have python entry point. We could basically open a socket connection for other programs to send R code in. Let me know what do you need in order to make it work.

[ManuelHentschel]

Thanks for the suggestion! Currently, the entire back and forth between the R package and the node wrapper is a bit convoluted, but in general, the following things I wasn't able to manage using just R:

• Give control back to the vscDebugger package whenever R hits a browser statement, error etc.. In principle this could be done using addTaskCallback and error handling, but that these seem to behave a bit weirdly when using browser()
• Write flow control commands (c, n, ....) to stdin.
• Manage direct user input to stdin, e.g. when using readLines()

Do you think these are straightforward to implement in radian?

[randy3k]

Perhaps let me explain how radian works. First, I assume that you understand how a REPL works

R: read from user input E: evaluate the input P: print the output L: loop

When R needs to read user input, it will call radian's callback read_console . There are a few things that we could do with this function.

1. before the function returns, we could run any arbitrary python or R code. (related to your 1 point)
2. If this function returns any text, R eventloop will evaluate the text. (related to your 2 and 3 point)

radian built in tcp server

The following is what I think should be doable. When radian starts, it opens a tcp server with a given port (something like radian --tcp 8123, or with an R function radian::run_tcp()). Other programs could connect to this port to communicate with radian (maybe using some kind of jsonrpc). A client could send requests to radian via the tcp connection, for example,

1. {"jsonrpc": "2.0", "method": "user_input", "params": "cat('hi')", "id": 1} cat('hi') will pass to R repl and the result will be printed in stdout

2. {"jsonrpc": "2.0", "method": "evaluate", "params": "1 + 3", "id": 1} The expression will be evaluated, and the reply message will be sent back to the client with the result.

3. {"jsonrpc": "2.0", "method": status", "id": 1} Status such as whether R is in debug prompt or regular prompt will be replied.

Something like this could add a lot of possibility to radian, including attaching a remote debugger to it.

[randy3k]

I think it is very similar to what you have done in vscDebugger. However, you did it in R which also means that the tcp server is interfering the R loop.

[randy3k]

Well, it seems that I might have misunderstood how DAP works. How do you handle the browse prompt in attach mode in the current implementation?

Edit: just saw https://github.com/ManuelHentschel/vscDebugger/pull/141 and it agrees with my understanding: attached mode is global space only.

[randy3k]

I have pushed a branch to radian. It allows you to define custom read console callback. I think it should fit your use case.

pip install git+https://github.com/randy3k/radian.git@callback

Simple use case. When read_console returns, the result is sent to R REPL for evaluation.

read_console <- function(message, add_history) {
    .radian.unregister_read_console()  # unregister it to avoid infinite loop
    return("1 + 3")
}

.radian.register_read_console(read_console)

For your user case. It will be more like this.

In user radian, we open a tcp server, and register a custom read console handler.

conn <- socketConnection(port = 8888, server = TRUE, blocking = TRUE)

read_console <- function(message, add_history) {
    while(TRUE) {
        line <- paste0(readLines(conn, n = 1), collapse = "\n")
        res <- jsonlite::fromJSON(line)
        if (res$method == "user_input") {
            return(res$data)
        } else if (res$method == "eval") {
            eval(parse(text = res$data), envir = .GlobalEnv)
        }
    }    
}

.radian.register_read_console(read_console)

Then in VSCode, you could make a tcp connection to interact with radian (I am using R to simulate the communication)

conn <- socketConnection(port = 8888)
writeLines(jsonlite::toJSON(list(method = "eval", data = "cat('hi\n')"), auto_unbox = TRUE), conn)
writeLines(jsonlite::toJSON(list(method = "user_input", data = "1 + 3"), auto_unbox = TRUE), conn)

I think it is possible to replicate this mechanism in regular R with some C code in macOS and Linux. However, it is not the case on Windows.

[randy3k]

Perhaps we should create a new package says for example dap under REditorSupport and start working a fresh and editor-agnostic implementation of dap there. It is in my opinion easier then modifying the current vscDebugger.

[ManuelHentschel]

Thanks for all the suggestions and the radian branch! I'll try to have a look at that branch later today.

Well, it seems that I might have misunderstood how DAP works. How do you handle the browse prompt in attach mode in the current implementation?

There are two cases where the attach mode can be used here:

• Calling .vsc.listen() in a normal R terminal. This will block the R input while a dap client is attached, but show the entire callstack etc.
• Calling .vsc.startWebsocket() to have the dap server run in the background (note that this funcion is misnamed as it does not actually start a websocket). This will not block R but will only have access to the global environment. I have been working on a rewrite of this feature on the branch tcl, but haven't had much time for this recently.

[sebffischer]

What is the status here? If there is anything simple that I can contribute (I don't have experience with these kinds of things) let me know :)

[ManuelHentschel]

I haven't had a chance to work on this in the last weeks, I'll try to get to it by the end of January!

[sebffischer]

Hey :) What is the status here?

[ManuelHentschel]

Hey, sorry for the inactivity here. In principle, the branch https://github.com/ManuelHentschel/vscDebugger/pull/176 might already work to a limited extent with different clients (setting breakpoints, variable view, and watches etc. might work, flow control definitely not).

I haven't been using R a lot recently, so I neglected this repo a bit. Next time I have to do a project in R I'll hopefully also work on this package more frequently. If you want to do anything yourself and have any questions in the meantime I'm happy to help, though :)

[sebffischer]

Someone started with a new implementation here https://github.com/dgkf/debugadapter/. But it has a lot of complexity that arise from circumventing the issues you also described here (see the issues in the repo). I would also be interested in contributing to the project if someone else with more experience can help in the design etc. , maybe we can structure it in such a way that it is either possible to use through radian and without radian. If we pool resources I am sure we can get something great going!

[sebffischer]

I guess if you are interested we could set up a discord or something for discussions. Or if there is already something like that for REditortSupport maybe we can discuss it there @randy3k ?

[ManuelHentschel]

It would be great if we found a clean way to solve the control-flow problems mentioned here and in https://github.com/dgkf/debugadapter/issues/4! Considering the difficulty of this task, I'm not sure how much time I'll be able to commit, but I'm always happy to talk if there is something to discuss.

[raffaem]

Any news on this?

Should we write to R-devel to propose the hook that we need?

I also use R within neovim, but debugging is a feature I lack

[ManuelHentschel]

Using https://github.com/posit-dev/ark might be a solution for this. For now they only support the positron front-end, but are planning to support others in the future.

[sebffischer]

some relevant discussion: https://floss.social/@lionel/112694030162713183

the browser hook that was added as part of the ark development might path the way for https://github.com/dgkf/debugadapter