sourcekit-lsp

sourcekit-lsp copied to clipboard

This implements #755 by introducing a new, non-standard request for fetching macro expansions, wrapping the corresponding SourceKitD request, SyntacticMacroExpansion.

Motivation

• There is precedent in rust-analyzer for such a request, see this document
• Using a dedicated request for this reduces the impedance mismatch between SourceKitD and the LSP layer
• Not imposing any particular presentation (as e.g. including it in hovers would) gives LSP clients freedom to present the expansion in a custom way.
  • In VSCode, for example, I would imagine that a read-only inline code editor, akin to the one shown when browsing references/definitions, might look nice:

Similar to what we did with inlay hints last year (#465), my hope is that a macro expansion request could eventually be upstreamed to LSP, given that such a request might be more generally useful to other languages too, as evidenced by the rust-analyzer implementation. This would then give clients full flexibility to implement a tailored UI too.

Feel free to leave some thoughts on whether this is the right approach to go with or alternative suggestions.

To do

• [x] Add an LSP request (sourcekit-lsp/macroExpansion) and support structures
  • Specifics are up for discussion, currently the request only takes a single position and returns a single expansion, should we support batch expansions, like SourceKitD does?
• [x] Add the SourceKit request SyntacticMacroExpansion
• [x] Add boilerplate to ToolchainLanguageServer and its implementations to support the request
• [x] Implement the actual request handler (SwiftLanguageServer.macroExpansion)
• [ ] Support nested expansions (perhaps customizable via a boolean flag in the request, e.g. nested: Bool or recursive: Bool?)
• [ ] Add unit tests (might require some more machinery to generate the macro plugin, similar to the upstream tests in SourceKit

See also

• SourceKit/upstream:
  • The SyntacticMacroExpansion test
  • The SyntacticMacroExpansion implementation
  • https://github.com/apple/swift/pull/66295
• VSCode:
  • https://github.com/swift-server/vscode-swift/pull/621

Thanks for starting the work on this @fwcd 🙏🏽

A couple of high-level thoughts, I haven’t looked at the actual code yet:

1. I know it’s very non-obvious, but the best way to expand the macro is to invoke the source.refactoring.kind.expand.macro refactoring. That’s what Xcode does as well. In case you are interested, you can view the sourcekitd request that Xcode sends by running Xcode as SOURCEKIT_SERVICE_LOG=1 /Applications/Xcode.app/Contents/MacOS/Xcode, opening a macro project, waiting for the log to calm down and then expanding a macro. You can set the log level to 2 to also see the responses
2. We should have the infrastructure to support nested macro expansions. The way this works on the sourcekitd side is that the macro expansion returns a buffer name for every edit. You can then expand macros inside the expansion buffer by setting key.primary_file to the originating source file and key.sourcefile to the buffer’s name (something like @__swiftmacro…). It’s probably best to see this in SourceKit service log in Xcode.
3. Do you know what the name in the rust-analyzer response is used for? Is it something that’s displayed to the user or could we use that for the buffer name?
4. @adam-fowler Would you be able to use a request like this to show the expanded macro in VSCode?

Those are already very helpful insights, thanks!

Most of the implementation in this PR is currently just standard boilerplate for adding a request, the actual logic is yet to be implemented. I will definitely have to take a closer look at how this refactoring works and how Xcode uses it, though.

Regarding 3 and 4: I've opened https://github.com/swift-server/vscode-swift/pull/621 with a super basic client implementation of the request that, admittedly, is still in dire need of a good UI. That thread also contains a short discussion of how rust-analyzer uses this request.

Oh, nice. I didn’t see the VSCode editor extension PR. ❤️

This actually ended up being a lot easier to implement than I thought thanks to the semantic refactoring!

Hi, before I open a new issue, does this address use of unknown directive '#Preview' errors? (#Preview being the built in macaro)

Hi, before I open a new issue, does this address use of unknown directive '#Preview' errors? (#Preview being the built in macaro)

No, that’s completely unrelated. It probably means that some compiler arguments are missing.

No, that’s completely unrelated. It probably means that some compiler arguments are missing.

In this case, there is no Package.swift, it's just me opening a folder with Swift files in it. We use Bazel.

Should the default arguments be adjusted?

Let’s discuss this in an issue to not spam this PR because it’s unrelated to this PR.

Support for macro expansions was merged in https://github.com/apple/sourcekit-lsp/pull/1436, so we don’t need this PR anymore. Thanks for your work on this PR @fwcd, it provided a lot of the design basis for https://github.com/apple/sourcekit-lsp/pull/1436.