Roadmap to BLISS

[davidanthoff]

This is an attempt to come up with a roadmap/plan for our cloud indexing, multi-environment and live environment enhancements. All of these features are pretty intertwined, so this is an attempt to come up with a sequence that allows us to ship intermediate versions of all of this without one gigantic switch-over. The idea is that we would ship after each phase in the released product.

@ZacLN does this make sense to you?

Phase I: version independent cache format

This would simply replace the currently used file format for our cache files with one that is not dependent on the Julia version, i.e. doesn't use serialization. My vote is we go with https://github.com/julia-vscode/SymbolServer.jl/pull/155 for now.

PRs and issues:

• [x] https://github.com/julia-vscode/SymbolServer.jl/pull/155

Phase II: cloud indexing

In this phase we would start to host pre-indexed cache files in the cloud and have the extension attempt to download the cache files from the cloud before it starts to index packages locally.

PRs and issues:

• [x] Figure out how we host the files in the cloud (https://github.com/julia-vscode/SymbolServer.jl/issues/200)
• [x] Write code that indexes everything in the registry and uploads it into the cloud (https://github.com/julia-vscode/SymbolServer.jl/pull/202)
• [ ] Index the existing registry (https://github.com/julia-vscode/symbolcache)
• [ ] Come up with some automatic way to keep the cloud cache up-to-date (CREATE ISSUE)
• [x] Add functionality in LS to download cache files from cloud and use those, if available (https://github.com/julia-vscode/SymbolServer.jl/pull/204)

Phase III: multi-environment

In this phase we would start to add support for multiple Julia environments to the LS. The broad idea is that we can have more than the one "global" active environment we currently support, i.e. that we can have a separate environment per folder in the workspace. Julia environments can be "concrete" (a Manifest.toml exists on disc) or "virtual" (for example the environment for the test folder).

In terms of Julia files, we essentially would have the existing logic that constructs trees of source files (based on include statements), then figure out for each tree to which of the env it belongs to.

One thing we might also want to handle in this phase is files that appear in more than one include-tree: I think right now we just more or less randomly pick one tree as "the one"? Maybe we should actually come up with a story where we for example then lint for each tree separately, i.e. try to detect if there is a problem in any tree. Not sure.

We should also add a file watcher for projects in this phase.

PRs and issues:

• [ ] Agree rules for how envs are loaded.
• [ ] Load multiple envs (SS)
• [ ] Associate each document tree with a single env (https://github.com/julia-vscode/StaticLint.jl/pull/251 and https://github.com/julia-vscode/LanguageServer.jl/pull/871)

Phase IV: live environments

In this phase we would add support for "live" dependencies in a Julia environment. Essentially this would mean one of two things:

1. if a package is deved in the env and the code for it is in the workspace, we don't use the symbol server logic for that package, but instead use the semantic information we got for that package from CSTParser and friends.
2. alternatively, we could also say that for any package that is deved, we always use the CSTParser to handle that. This would mean that in most situations, the symbol server indexing would no longer run at all on user computers (tagged packages are handled by the cloud cache, deved versions are live parsed). On the other hand we would lose some richness... Not sure about this, but I think the scenario where someone has something deved that is not in the workspace but maybe is being edited in another VS Code window is probably common, and this approach would at least mean we pick up edits more quickly.

PRs and issues:

• https://github.com/julia-vscode/LanguageServer.jl/pull/562

[ZacLN]

• [ ] Agree rules for how envs are loaded.

So, thoughts on options for the above:

• Allow users to set the env for each project tree based on some vscode-side configuration.
• Attempt to automatically detect the env (i.e. search for a project file)