Support custom stubs

[bartekpacia]

hey! this looks like such a cool project, thanks so much for developing it as open-source.

I am wondering if {there is support/there could be support} for adding types and docs to built-in functions (defined by the program that embeds a Starlark interpreter).

I found that another Starlark LSP supports this (although that LSP seems focused on only supporting Tiltfiles and I couldn't get it to work with their VSCode extension). But what that other LSP allows is something like:

starlark-lsp start --address ":9999" --builtin-paths stubs.py

whereas stubs.py is, for example:

def foo(name: str) -> bool:
  """
  This is sample documentation of the foo function.
  """
  pass

I read the README of this repo a few times and looked for existing issues but didn't find a similar issue. Sorry if there is!

Also, I'm not sure if this project is focused solely on developing an LSP for Bazel or a more generic LSP for Starlark. Clarifiying this in the README would be cool.

[BlueishSapphire]

This feature would be very helpful! Many dialects of Starlark exist and could use LSP support, and this project seems to be the best Starlark LSP at the moment.

I attempted to load some custom builtins by making a custom protobuf json file in ./crates/starpls_bazel/data and messing around with the build system, but to no avail. It does seem like the best way to support custom stubs would be through the existing builtins protobuf system, though. I imagine the easiest way to do this would be to extend the code to allow adding custom builtin definitions in files such as /crates/starpls_bazel/data/my_dialect.builtins.json and have those symbols loaded globally. This wouldn't be perfect, but it would at least allow people to hack together dialect support.

[withered-magic]

Thanks for the pointer to that LSP, I haven't seen that before!

I think that the stub file makes sense as a means for defining builtins, will look further into this since I have a similar use case for this at $dayjob. The files under ./crates/starpls_bazel/data were definitely more meant as a workaround for bzlmod globals etc. not being included in the builtins proto, but it should be straightforward enough to allow defining builtins in either format for now depending on user preference, and then maybe standardize on one later

[albertocavalcante]

Hi all,

I would love to give it a try at tackling this, as I'm eager for this feature :)

But first, would like to check if there have been any thoughts on what the implementation should look like.

If we take the tilt-dev LSP implementation as a reference, we would have to introduce Python parsing capabilities to starpls. They use tree-sitter for that (https://github.com/tilt-dev/starlark-lsp/blob/main/pkg/query/parse.go#L11). In Rust, maybe https://github.com/astral-sh/ruff/tree/main/crates/ruff_python_parser could be an option as well.

That said, is this how we should proceed?

I don't think Starlark itself would be expressive enough to define stubs, as it lacks classes. (sample to compare: https://github.com/tilt-dev/tilt/blob/master/internal/tiltfile/api/init.py)

Another possible approach is to leverage the approach taken in the starpls_bazel crate, as suggested by @BlueishSapphire.

I gave it a try just to see a possible visualization of the implementation: (https://github.com/albertocavalcante/fork-starpls/blob/feature/extensible-dialect-support/DIALECT_EXTENSION_SPEC.md#json-schema). This is mostly AI-generated, so sorry for the garbage: https://github.com/withered-magic/starpls/compare/main...albertocavalcante:fork-starpls:feature/extensible-dialect-support.

Another approach I've just seen taken by Aspect CLI and their .axl Starlark dialect is to base a crate on the https://crates.io/crates/starlark_lsp. (https://github.com/aspect-build/aspect-cli/tree/main/crates/axl-lsp). cc: @gregmagolan @alexeagle

Which makes me think if it's also possible to either have starpls as a crate and extend it (benefits: a single LSP that understands both Starlark and Bazel specific semantics/built-ins + all the good feature stuff starpls has) - or maybe even more ambitious - allowing starpls to have plugins that extend its behavior. (The AI slop above is an attempt to make a plugin definition in JSON - not great though.)

Also - you know - the Bazel VSCode Plugin is LSP-agnostic and can only trigger a single LSP, so just relying on a single one for everything would be nice, or, in my use case, creating a custom extension for Starlark itself may also seem appealing sometimes.

Sorry for the long text. I would appreciate any thoughts on how we should land this.

Thank you!