Confused about `remote_eval` and string literals

[adigitoleo]

It's really just a question, I'm not sure why this doesn't work. Today I was looking for a way to have a daemon running JuliaFormatter so I can occasionally quickly reformat messy files. I've got something that works when using DaemonMode.jl but not using this package.

Starting the server

DaemonMode.jl version:

jl --startup-file=no -e 'using JuliaFormatter,DaemonMode; serve()'

RemoteREPL.jl version:

jl --startup-file=no -e 'using JuliaFormatter,RemoteREPL; serve_repl()'

Client (code formatter script)

DaemonMode.jl version:

julia --startup-file=no -e "using DaemonMode; runexpr(\"\"\"format_file(\"$1\")\"\"\")"

RemoteREPL.jl version:

julia --startup-file=no -e "using RemoteREPL; RemoteREPL.remote_eval(\"\"\"format_file(\"$1\")\"\"\")"

Test file with bad formatting

begin
"""This is a badly formatted file"""
    a = 4
b = 2

c = (   "foo",
"bar")
end

The DaemonMode.jl version works as expected. The RemoteREPL.jl version connects to the server and then exits, but the code stays unformatted. From what I can see, DaemonMode just printlns the string to the socket, but here you have some more processing going on. I wonder if that is causing an issue with the triple-quoted strings or something?

[adigitoleo]

The weird part is that I don't see any errors, so I'm not sure what's happening.

[c42f]

I think your confusion here is due to the environment in which DaemonMode vs RemoteREPL evaluate their commands.

RemoteREPL is server-centred by default. This means that code

• Executes in the Main module by default
• Uses the current working directory of the server
• Does not hijack the server's stdout and stderr

DaemonMode is client-centred by default. This means that code

• Executes code in a temporary module
• Runs code in the working directory of the client
• Hijacks the server's stdout and stderr to send to the client

With this in mind, DaemonMode is more suited for your use case here. On the other hand, you could have debugged this issue interactively with RemoteREPL (it's a REPL, after all!). You might have seen something like the following:

chris@xyz:~/tmp$ pwd
/home/chris/tmp

chris@xyz:~/tmp$ julia -q

julia> using RemoteREPL

julia> connect_repl()
REPL mode remote_repl initialized. Press > to enter and backspace to exit.

julia@localhost> format_file("badfile1.jl")
ERROR: SystemError: opening file "badfile1.jl": No such file or directory
Stacktrace:
  [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
...

julia@localhost> pwd()
"/home/chris"

[c42f]

For your use case, the following should work ok. The trick is to compute the abspath on the client side before sending to the server:

julia --startup-file=no -e 'using RemoteREPL; path=abspath(ARGS[1]); RemoteREPL.remote_eval("format_file(\"$path\")")' -- badfile.jl

The fact that remote_eval didn't show you an exception is a bug, I think.

I also opened #30 which would help with error reporting if JuliaFormatter writes to stdout.

[adigitoleo]

Thanks for explaining, it makes sense. I'll stick to using DaemonMode for this for now, it also seems to be slightly faster (haven't benchmarked, just a hunch).

Thoughts on exporting remote_eval?

I was reading about sockets and streams the other day because it's all quite new to me, and it seems like it could be nice to have DaemonMode or this package falling back to unix sockets instead of TCP if both server and client are on the same (unix) host (better performance, possibly better secrity?). I wonder if you've tried using unix sockets before, or if you have some insights about that.

[c42f]

it also seems to be slightly faster (haven't benchmarked, just a hunch)

Probably DaemonMode has lower startup time due to precompiling some things? Or perhaps because they don't need an ssh tunnel for transport over the open internet! Better startup time would be nice to have in RemoteREPL too, but it hasn't been a big priority yet as the use cases are different.

Thoughts on exporting remote_eval?

I think that would be fine, though I somehow feel it's not a great API :-)

falling back to unix sockets instead of TCP

Yes this definitely makes sense and I've been vaguely planning to do it at some point. Using unix domain sockets or windows named pipes would make sense on localhost and the Sockets stdlib even has a function to make it easy:

    listen(path::AbstractString) -> PipeServer

Create and listen on a named pipe / UNIX domain socket.

The main advantage is that it makes RemoteREPL secure to use on multi-user servers (at least on unix where you can use filesystem permissions to protect against other users on the system. I'm not sure about windows named pipes.). It should also help a bit with startup time.

[c42f]

I opened an issue about this https://github.com/c42f/RemoteREPL.jl/issues/31

[c42f]

By the way, startup time for the client can be greatly reduced using the julia-r script here:

https://github.com/c42f/RemoteREPL.jl/blob/main/bin/julia-r

This sets some Julia options to make startup almost instant.