"Import could not be resolved" after installing the package

[ibobak]

Type: Bug

The problem is that pylance continues to report error even after package installation.

The sequence of action was as on the picture. Steps to reproduce:

1. install anaconda
2. open attached notebook, run it under base environment. You will get an error in the last cell because the package is missing
3. pip install gap-stat
4. run the last cell. it works. but pylance is still reporting the problem (which does not exist any more)

The notebook is attached (rename to ipynb): kmeans.ipynb.txt

Extension version: 2024.6.1 VS Code version: Code 1.90.2 (5437499feb04f7a586f677b155b039bc2b3669eb, 2024-06-18T22:33:48.698Z) OS version: Linux x64 6.4.6-060406-generic Modes:

System Info

Item	Value
CPUs	Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz (88 x 1197)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	1, 2, 1
Memory (System)	251.76GB (241.06GB free)
Process Argv	--crash-reporter-id 27d42247-63fb-4d9e-9cb0-87d9974843dc
Screen Reader	no
VM	50%
DESKTOP_SESSION	ubuntu-xorg
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu-xorg
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
jchc7451:31067544
chatpanelt:31048053
dsvsc021:30996838
9c06g630:31013171
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
legacy_priority:31077528

[heejaechang]

will it go away if you close and re-open the notebook? or if you reload vsocde?

[ibobak]

yes, it goes away if I close and open the notebook.

But this is not an option and here is why: I have Spark context running in this notebook, and many initialized pandas dataframes, which are quite hard to calculate. And if I close the notebook and re-open it again, I will have to re-initialize the Spark app, recalculate of my dataframes, etc., which is a hard routine and takes a lot of time. Therefore, keeping a notebook running (even with false errors from pylance) is better than closing and re-opening of it.

But anyway, it would be good to have a command in Ctrl + Shift + P named "Pylance: restart scanning of files/notebooks" which would just do the job

[ibobak]

I don't know how the guys from PyCharm team did it, but they re-scan of all installed packages of the current environment just as soon as the new package arrives. It would be good to have this here too.

[ibobak]

I see that this bug in "needs repro" status. I don't know if you noticed this: I provided a notebook file in my original message, together with steps to reproduce.

[rchiodo]

This could be this issue here: https://github.com/microsoft/vscode-jupyter/issues/15965

The root cause of the problem is you change your kernel and we (Pylance) never get told about it. So we think the current interpreter is a different one.

That's why Heejae asked if it went away if you closed and opened the notebook, we get the initial interpreter correctly then.

[rchiodo]

The log from the Pylance Language Server should tell us which interpreter we think is being used. (If it's the VS code issue I mentioned, it won't match the kernel).

[StellaHuang95]

Just following up on the issue. Does it still repro for you? If so, could you please provide more logs as outlined in our troubleshooting guide here? Thanks.

[ibobak]

I've tried with the latest version of VS Code and it still reproduces: https://youtu.be/TkXpookpuDg (switch on the audio, please)

Current versions:

VS Code version: Code 1.91.1 (f1e16e1e6214d7c44d078b1f0607b2388f29d729, 2024-07-09T22:08:12.169Z) OS version: Linux x64 6.4.6-060406-generic Modes:

System Info

Item	Value
CPUs	Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz (88 x 1297)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	2, 2, 1
Memory (System)	251.76GB (234.81GB free)
Process Argv	--crash-reporter-id 27d42247-63fb-4d9e-9cb0-87d9974843dc
Screen Reader	no
VM	50%
DESKTOP_SESSION	ubuntu-xorg
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu-xorg
XDG_SESSION_TYPE	x11

Extensions (55)

Extension	Author (truncated)	Version
numbered-bookmarks	ale	8.5.0
project-manager	ale	12.8.0
zoomer	ant	0.3.1
vscode-django	bat	1.15.0
docs-view	bie	0.1.0
path-intellisense	chr	2.9.0
gitignore	cod	0.9.0
git-extension-pack	don	0.1.3
githistory	don	0.6.20
python-environment-manager	don	1.2.4
python-extension-pack	don	1.7.0
gitlens	eam	15.2.0
code-runner	for	0.12.2
vscode-edit-csv	jan	0.9.2
vscode-icon-theme	jtl	1.6.6
vsc-python-indent	Kev	1.18.0
vscode-checkpoints	mic	1.3.3
vscode-docker	ms-	1.29.1
csdevkit	ms-	1.8.14
csharp	ms-	2.34.12
vscode-dotnet-runtime	ms-	2.1.1
vscodeintellicode-csharp	ms-	2.1.11
black-formatter	ms-	2024.2.0
debugpy	ms-	2024.8.0
pylint	ms-	2023.10.1
python	ms-	2024.10.0
vscode-pylance	ms-	2024.7.1
datawrangler	ms-	1.4.2
jupyter	ms-	2024.6.0
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.18
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-powertoys	ms-	0.1.1
vscode-jupyter-slideshow	ms-	0.1.6
remote-containers	ms-	0.375.1
remote-ssh	ms-	0.112.0
remote-ssh-edit	ms-	0.86.0
remote-wsl	ms-	0.88.2
vscode-remote-extensionpack	ms-	0.25.0
remote-explorer	ms-	0.4.3
remote-server	ms-	1.5.2
vs-keybindings	ms-	0.2.1
one-dark-theme	msk	1.14.2
autodocstring	njp	0.6.1
git-file-history	pom	1.0.1
inline-sql-syntax	quf	2.16.0
gpack	Sey	2.0.0
markdown-preview-enhanced	shd	0.8.13
intellicode-api-usage-examples	Vis	0.2.8
vscodeintellicode	Vis	1.3.1
vscode-conventional-commits	viv	1.25.0
jinja	who	0.0.8
markdown-all-in-one	yzh	3.6.2
material-theme	zhu	3.17.2
vscode-open-in-github	ziy	1.3.6

(11 theme extensions excluded)

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
9c06g630:31013171
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
legacy_priority:31094926

[rchiodo]

We'd still need the logs. It would show if we actually are using the correct interpreter for the kernel selected. My guess is that this is the VS code issue which is caused by the kernel being selected having the same python version.

[ibobak]

logs.txt

And here is how I reproduced this situation and recorded those logs: https://www.youtube.com/watch?v=AjeRVlLrNcA

As you may have noticed on my video, there are no actions performed (based on the logs) after I installed that missing package. If it was, I would see some messages in my log just AFTER I installed it.

[rchiodo]

From watching the video and looking at the log it seems like it's not the VS code issue. It's more like we're not getting a file system events when you install the gap_statistic module.

According to the log we should be watching these folders here:

2024-07-18 19:35:58.602 [info] [Info  - 7:35:58 PM] (151781) Adding fs watcher for library directories:
 file:///opt/anaconda3/lib/python3.10
file:///opt/anaconda3/lib/python3.10/lib-dynload
file:///opt/anaconda3/lib/python3.10/site-packages
file:///opt/anaconda3/lib/python3.10/site-packages/PyQt5_sip-12.11.0-py3.10-linux-x86_64.egg
file:///opt/anaconda3/lib/python3.10/site-packages/mpmath-1.2.1-py3.10.egg

If you print out gap_statistic.__file__, where does it say it comes from?

[ibobak]

It will be in /opt/anaconda3/lib/python3.10/site-packages/gap_statistic/init.py

[rchiodo]

If you touch that file (or maybe just add a file in that directory), do you get any fs watcher output in the Python Language Server?

For example, I get this if I add a file in my site-packages folder:

2024-07-18 10:45:43.131 [info] [Info  - 10:45:43 AM] (26484) LibraryFile: Received fs event 'add' for path 'c:\Users\rchiodo\AppData\Local\anaconda3\envs\Pesnet\Lib\foo.py'
2024-07-18 10:45:43.131 [info] [Info  - 10:45:43 AM] (26484) LibraryFile: Received fs event 'add' for path 'c:\Users\rchiodo\AppData\Local\anaconda3\envs\Pesnet\Lib\foo.py'
2024-07-18 10:45:43.131 [info] [Info  - 10:45:43 AM] (26484) LibraryFile: Received fs event 'add' for path 'c:\Users\rchiodo\AppData\Local\anaconda3\envs\Pesnet\Lib\foo.py'
2024-07-18 10:45:43.131 [info] [Info  - 10:45:43 AM] (26484) LibraryFile: Received fs event 'add' for path 'c:\Users\rchiodo\AppData\Local\anaconda3\envs\Pesnet\Lib\foo.py'

[ibobak]

I tried to edit the file, add lines, save it, creating a new file in that directory - nothing helped: https://youtu.be/P7cBw8YPSTE I know for sure that I launched multiple times the command

chmod a+rw -R /opt/anaconda3

So there are permissions to read/write from that directory. But the question is: are there permissions for tracking file changes? How to find out in Linux? Could you give some script/code which tracks for changes, so that I will run it and tell it to you?

[rchiodo]

Sorry I'm entirely sure how we track changes. It's part of the VS code API, we just tell it which files to track. I believe it uses something called https://www.npmjs.com/package/chokidar. You could potentially write a javascript file that uses chokidar as in the npm examples and then try editing a file that's being watched?

[ibobak]

I reproduced a small app in js (with usage of chokidar) and it works just fine. But there is one catch.

The code of my js file:

// Import the chokidar package
const chokidar = require('chokidar');

// Define the directory you want to watch
const directoryToWatch = '/opt/anaconda3/lib/python3.10/site-packages/gap_statistic';

// Initialize the watcher
const watcher = chokidar.watch(directoryToWatch, {
  ignored: /(^|[\/\\])\../, // ignore dotfiles
  persistent: true,
});

// Define event handlers
watcher
  .on('add', path => console.log(`File ${path} has been added`))
  .on('change', path => console.log(`File ${path} has been changed`))
  .on('unlink', path => console.log(`File ${path} has been removed`))
  .on('addDir', path => console.log(`Directory ${path} has been added`))
  .on('unlinkDir', path => console.log(`Directory ${path} has been removed`))
  .on('error', error => console.log(`Watcher error: ${error}`))
  .on('ready', () => console.log('Initial scan complete. Ready for changes'))
  .on('raw', (event, path, details) => {
    // This event should be used only if you need raw event details
    console.log('Raw event info:', event, path, details);
  });

console.log(`Watching for changes in ${directoryToWatch}`);

then I am running it as

node ./1.js

and here is what I see when I add a new file to this directory:

So, it looks like my simple app is working fine. But please, look how I initialized it:

const directoryToWatch = '/opt/anaconda3/lib/python3.10/site-packages/gap_statistic';

Let us make one change:

const directoryToWatch = '/opt/anaconda3/lib/python3.10/site-packages';

If this is done and node ./1.js is launched, then tracking for changes also has place, but it slowly enumerates all files (there are a lot of them in site-packages because this is anaconda's base env). However, after adding a file I see this:

So, the question is this: are you sure you are using chokidar correctly and not facing a similar issue?

[rchiodo]

So, the question is this: are you sure you are using chokidar correctly and not facing a similar issue?

That same error you got should show up in the VS code output somewhere if there was the same problem. Not the pylance output (as we don't create the file watchers) but likely this one:

See this issue we had a while back. You can control this problem using globs in VS code: https://github.com/microsoft/pylance-release/issues/5639

[rchiodo]

This might help too: https://code.visualstudio.com/docs/setup/linux#_visual-studio-code-is-unable-to-watch-for-file-changes-in-this-large-workspace-error-enospc

[ibobak]

In the extension host I see this: logs_extension_host.txt

It doesn't look I am having any issues with number of files. I mentioned this issue related to my own js script, but that doesn't mean that VS Code does monitoring the same way I did when I wrote this little script with the help of CharGPT (because I am not a js developer at all).

So, that was just my guess, but no confirmation in logs. And the question is: how to find the reason why it doesn't work?

There is a second question: is it really difficult to make me a command "rescan libraries" in Ctrl+Shift+P? So that after library installation, I will just run this command, and pylance will re-index everything. Please....

[rchiodo]

Technically you could just use "Python : Restart Language Server". It would have the same effect.

[rchiodo]

And the question is: how to find the reason why it doesn't work?

We'd have to repro it in house to see if we could suss out why the file watchers are not actually watching. It's likely the limit thing, but I'm not entirely sure where that error crops up. You could also try upping the limit as specified in that link I sent to see if that fixed it.

or just restart the language server.

[ibobak]

Rich, thank you very much for the command name - now I will know that the name is "Restart Language Server" :)

If you know how to dump all logs using one command (if there is any) from all vs code subsystems and just zip them all an give to you - I will do this. This may help to find the reason.

Currently I know only how to see them separately in the bottom window, and that is why I did not find anything about limits, so there is still no proof.

[rchiodo]

Yeah it might not even be VS code that's outputting the error. Some other system might. Not sure how they even detect it.

But you can get to where all the logs are here:

Developer : Open Extension Logs Folder

You might have to also set VS code's log level too. That's with the Developer: Set Log Level command.

[github-actions[bot]]

This issue has been closed automatically because it needs more information and has not had recent activity. If the issue still persists, please reopen with the information requested. Thanks.