julia-vscode
julia-vscode copied to clipboard
julia-vscode
Support DocStringExtensions
Ideally the DocStringExtensions variables are expanded automatically and markdown format is respected. The Julia REPL can handle it properly.
Note that it's not working ONLY within the same package that I'm developing. If DocStringExtensions is used in a third package, and I'm just using that package, then the doc is displayed properly.
My gut feeling is that we probably only want to support this once these features are moved into Base...
In some sense it already is in Base though. The docsystem has an (undocumented) API to add a custom hook to process docstrings (that's how DocStringExtensions works). The "user" (e.g. julia-vscode) should fetch the processed docstrings.
But I guess it already does that, since it actually works for "third packages". Just that for the docstrings that are updated in VS Code the custom hook doesn't get run on the new docstring or something along those lines?
Yeah, for third party packages it works because we load the package and then use the normal doc extraction feature.
But for code that the user edits live, we extract the doc string simply by parsing the code file, with no additional processing. We could probably support some pre-defined things if they are official enough, but I don't see a reasonable design that would run custom hooks in that scenario...
Do the updated docstrings get passed on to the docsystem? If yes, could you not extract the docstrings also in this instance? Or.. do you actually never run the edited code? (Pardon my ignorance -- I haven't actually ever used julia-vscode)
We have both a runtime process (in which the user can run code, either via the REPL or inline execution, or with Revise) and a server process that never comes into contact with user code. The latter parses user code and extracts docs from the CST, so no, those never come into contact with the docsystem.
I understand. What would be a sensible implementation if DocStringExtensions never gets into Base? I am imagining that code like the callme function above could be evaluated in a made-up module, just for extracting the interpolated doc string?
module TempModule1245
using DocStringExtensions
"""
$(TYPEDSIGNATURES)
Just call me!
"""
function callme()
# intentionally blank
end
end
We generally never execute user code in the language server process (because it could crash it). So if we wanted to run something, it would have to be in a different process, like what we do with the symbol server process. That would be a lot of overhead...
The alternative would be to teach the LS about particular syntax, like the TYPEDSIGNATURES for example, presumably there aren't that many and one could just reimplement them in the LS, and not execute any code? But I think the likelihood of us adding that to the LS would really be higher if it was in base and officially part of the doc system...
I don't think I'm suggesting executing user code though. Knowing that there's a doc string above a function definition, we can just evaluate the doc string with an empty function with the same name. Same thing for structs.
Come to think of it - for a MVP (minimum viable product), I would actually be fine if the string is parsed and shown as markdown rather than a regular string. The \n's are more annoying as majority of my doc strings have multiple lines, sometimes with bullets, etc. and they're not formatted properly.
we can just evaluate the doc string
As soon as we do that, we are executing user code, though...
I can see your point :-) but I was just thinking about what can go wrong. The eval's are done using DocStringExtensions and it does generate the text from the function's signature. Could that really crash the process?
Users can presumably run arbitrary code inside $(), it is just standard string interpolation, right?
It is. Would it be safe if we make it resolve to only DocStringExtensions variables and disallow arbitrary expressions?
I think some sort of whitelist approach would probably be safest: only allow a few hand-selected things like $(TYPEDSIGNATURES). But no matter how, we would have to include special casing for DocStringExtensions.jl in the language server code base, and we try quite hard to not do that in general...
How about we just settle with the MVP that I described above then i.e. just make sure that it's parsed as a markdown string rather than a regular string. That honestly feels more like a bug.
I think this approach would be reasonable even if DocStringExtensions never gets into Base.
I can also work on this if you can just point me to the right direction 😄 I'm quite confused about where to find the code among all the vscode repos...
Yes, totally agreed, we should definitely treat it as markdown! I have no idea where to change that, @ZacLN is the king of all of this stuff :)
