Offer different pylance modes to enable light vs full IntelliSense experience

[luabud]

Right now we can tweak settings such as enabling/disabling indexing, changing indexing limit, packageIndexDepths etc to decrease/increase the amount of resources that are used (https://github.com/microsoft/pylance-release/wiki/Pylance-Configuration-Tips), which also impacts the type of information we are able to offer as part of our IntelliSense features. It'd be nice if we had different language service "modes" (e.g. lightweight - only supports editor related features with minimum stuff enabled, default for what we do now, and full which indexes as much as we can and enable expensive features on by default) that users can change depending on what they want and need.

[heejaechang]

this will require some groundwork before doing this work.

1. resolve indexing perf regression we are having (or improve it better than before)
2. support requirements.txt and pyproject.toml to limit how much we index, show in completion and etc in full mode (otherwise, it will be too much)
3. support alias in user files's indexing (we have cut it due to perf issue)

[heejaechang]

1. resolve indexing perf regression we are having (or improve it better than before)

this has been handled by https://github.com/microsoft/pyrx/pull/5007 and https://github.com/microsoft/pyright/pull/7652

[luabud]

One idea inspired by cases like https://github.com/microsoft/pylance-release/issues/6055 is to have lightweight mode disable workspace wide features, and offer open file support only

[heejaechang]

there is another issue that UX could be improved by light mode - https://github.com/microsoft/pylance-release/issues/5881

[heejaechang]

another PR related to this issue - https://github.com/microsoft/pyrx/pull/5437

[nineteendo]

Did you mean: https://github.com/microsoft/pyright/issues/5437?

[heejaechang]

@nineteendo no, that's not related to this work. might be related to something @debonte working on that's about doc annotation?

[nineteendo]

Is microsoft/pyrx just a private repo?

[rchiodo]

Is microsoft/pyrx just a private repo?

Yes it's the source location for Pylance. Which is based on Pyright.

[heejaechang]

we should enhance indexing experience such as this - https://github.com/microsoft/pylance-release/issues/6209 - to make full mode indexing works better.

[noklam]

What's the progress on this issue? As a library/extension developer, I would like to set the index depth automatically.

I've read most of the related thread, I think reading requirements.txt/pyproject.toml is a sensible default (And the source code of current project of course). This would solve my problem and I don't need a workaround.

As temporary workaround, If I want to change this settings in a VS Code extension, is it possible? I may able to override the default but what I want is merge with the exist PyLance settings and I am not sure how it will works.

For example, if an user have default Flask and Django will index with depth 2, when he/she also installed Kedro extension, I'd like it to add kedro index with depth 2 automatically. Is something like this possible?

Personally, I think the default with depth 1 index has some conflict with Python import mechanism. In term of best practice, library generally avoid putting too much on the top level, because this will cause all submodule being imported automatically, so we aim to decrease the API surface as much as possible.

i.e. import a.b.c.d will import a, a.b, a.b.c and a.b.c.d

This is however causing friction when used with VSCode because one will certainly appreciate the help of the IDE. This mean that if I am a library developer and I want my users have the best auto-completion experience, I am forced to put all the API to the top level.

[heejaechang]

@noklam I am working on it. if you have suggestion, leaving it here will help us designing the feature.