pylance-release
pylance-release copied to clipboard
microsoft
pylance is too slow when using sklearn. Checking and Analyzing are repeated every time when the file saved ("Long operation: checking:")
Environment data
- Language Server version: 2023.2.30
- OS and version: Mac OS Ventura 13.2.1, VS code 1.75.1
- Python version (& distribution if applicable, e.g. Anaconda): Python 3.10
Code Snippet
- Reproduce code
import numpy as np
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA
class sklearn_test(object):
def process(self, reduction_method = 'pca'):
if reduction_method == 'pca' :
self.reduction_method = PCA(n_components=3, random_state=2023)
elif reduction_method == 'tsne' :
self.reduction_method = TSNE(n_components=3, random_state=2023)
value = np.random.random((10,10))
reduced_values = self.reduction_method.fit_transform(value)
return reduced_values
if __name__ == "__main__":
a = sklearn_test()
reduced_values = a.process()
print(reduced_values)
b = dict()
for key, value in b.items():
print("hi")
- conda requirements.txt
name: debug
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _openmp_mutex=5.1=1_gnu
- bzip2=1.0.8=h7b6447c_0
- ca-certificates=2023.01.10=h06a4308_0
- certifi=2022.12.7=py310h06a4308_0
- ld_impl_linux-64=2.38=h1181459_1
- libffi=3.4.2=h6a678d5_6
- libgcc-ng=11.2.0=h1234567_1
- libgomp=11.2.0=h1234567_1
- libstdcxx-ng=11.2.0=h1234567_1
- libuuid=1.41.5=h5eee18b_0
- ncurses=6.4=h6a678d5_0
- openssl=1.1.1t=h7f8727e_0
- pip=22.3.1=py310h06a4308_0
- python=3.10.9=h7a1cb2a_0
- readline=8.2=h5eee18b_0
- setuptools=65.6.3=py310h06a4308_0
- sqlite=3.40.1=h5082296_0
- tk=8.6.12=h1ccaba5_0
- tzdata=2022g=h04d1e81_0
- wheel=0.38.4=py310h06a4308_0
- xz=5.2.10=h5eee18b_1
- zlib=1.2.13=h5eee18b_0
- pip:
- joblib==1.2.0
- numpy==1.24.2
- scikit-learn==1.2.1
- scipy==1.10.1
- threadpoolctl==3.1.0
prefix: /anaconda/envs/debug
Repro Steps
- Create conda env according to conda requirements.txt
- copy text to conda_requirements.txt and paste the command to terminal.
conda env create -f conda_requirements.txt
- Activate conda env and open the reproduce code with VSCode
Expected behavior
- comment out line 12
Actual behavior
- include line 12
Logs
[Info - 7:17:42 AM] (1460942) SourceFile: Received fs event 'change' for path '/home/cyjun0304/Desktop/debug_pylance/main.py'
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
[Info - 7:17:50 AM] (1460942) SourceFile: Received fs event 'change' for path '/home/cyjun0304/Desktop/debug_pylance/main.py'
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5562ms)
[Info - 7:17:55 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5562ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5564ms)
[Info - 7:17:55 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5564ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] indexing: /home/cyjun0304/Desktop/debug_pylance/main.py [found 6] (0ms)
(1460942) Indexing Done: /home/cyjun0304/Desktop/debug_pylance/main.py
(1460942) Background analysis message: getSemanticTokens delta
(1460942) [BG(1)] getSemanticTokens delta previousResultId:1676877461965 at /home/cyjun0304/Desktop/debug_pylance/main.py (3ms)
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:9 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:9 [found 59 items] (3ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 5 ms, Memory increase: -179 KB.
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:5 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:5 [found 59 items] (3ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 3 ms, Memory increase: 658 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:9 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:9 [found 46 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 4 ms, Memory increase: -232 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:10 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:10 [found 14 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1537 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:11 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:22:11 [found 5 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 3 ms, Memory increase: -309 KB.
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] parsing: /tmp/pyright-1460942-y29NPBM1pJzz/builtins-1460942-B6MaD7MK2ft9-.py [fs read 1ms] (78ms)
(1460942) [FG] binding: /tmp/pyright-1460942-y29NPBM1pJzz/builtins-1460942-B6MaD7MK2ft9-.py (14ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:5 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:5 [found 23 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1582 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:6 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:6 [found 15 items] (1ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1572 KB.
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:9 [found null items] (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:11 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:11 [found null items] (1ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:14 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:14 [found 12 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1617 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:15 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:15 [found 6 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 3 ms, Memory increase: 1615 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:16 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:16 [found 5 items] (1ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1609 KB.
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:20 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:20 [found 99 items] (1ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1630 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:21 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:21 [found 50 items] (3ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 3 ms, Memory increase: 1604 KB.
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5165ms)
[Info - 7:18:21 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5165ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5167ms)
[Info - 7:18:21 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5167ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:23 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:23 [found 60 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 3 ms, Memory increase: -6087 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:24 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:24 [found 43 items] (20ms)
(1460942)
Lookback tokens:
(,n_components,=,NUM_LIT,,,random_state,=,NUM_LIT,),\n,value,=,np,.,random,.,random,(,(,NUM_LIT,,,NUM_LIT,),),\n,reduced_values,=,self,.,reduction_method,.,fit_transform,(,value,),\n,return,reduced_values,\n,if,__name__,==,STR_LIT,:,\n,a,=,sklearn_test,(,),\n,reduced_values,=,a,.,process,(,),\n,print,(,reduced_values,),\n,b,=,dict,(,),\n,for,key,,,value,in,b,.,items,(,),:,\n,print,(,STR_LIT,),\n,c,=,dict,(,),\n,for,key,,,value,in,dict,.
(1460942) Recommendations: items, iteritems, keys, _get_kwargs, __dict__, environ, viewitems, body, cells, values
(1460942) Time taken to get recommendations: 58 ms, Memory increase: -5601 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:25 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:25 [found 16 items] (16ms)
(1460942)
Lookback tokens:
(,n_components,=,NUM_LIT,,,random_state,=,NUM_LIT,),\n,value,=,np,.,random,.,random,(,(,NUM_LIT,,,NUM_LIT,),),\n,reduced_values,=,self,.,reduction_method,.,fit_transform,(,value,),\n,return,reduced_values,\n,if,__name__,==,STR_LIT,:,\n,a,=,sklearn_test,(,),\n,reduced_values,=,a,.,process,(,),\n,print,(,reduced_values,),\n,b,=,dict,(,),\n,for,key,,,value,in,b,.,items,(,),:,\n,print,(,STR_LIT,),\n,c,=,dict,(,),\n,for,key,,,value,in,dict,.
(1460942) Recommendations: items, iteritems, keys, _get_kwargs, __dict__, environ, viewitems, body, cells, values
(1460942) Time taken to get recommendations: 61 ms, Memory increase: 1982 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:27 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:23:27 [found 8 items] (19ms)
(1460942)
Lookback tokens:
(,n_components,=,NUM_LIT,,,random_state,=,NUM_LIT,),\n,value,=,np,.,random,.,random,(,(,NUM_LIT,,,NUM_LIT,),),\n,reduced_values,=,self,.,reduction_method,.,fit_transform,(,value,),\n,return,reduced_values,\n,if,__name__,==,STR_LIT,:,\n,a,=,sklearn_test,(,),\n,reduced_values,=,a,.,process,(,),\n,print,(,reduced_values,),\n,b,=,dict,(,),\n,for,key,,,value,in,b,.,items,(,),:,\n,print,(,STR_LIT,),\n,c,=,dict,(,),\n,for,key,,,value,in,dict,.
(1460942) Recommendations: items, iteritems, keys, _get_kwargs, __dict__, environ, viewitems, body, cells, values
(1460942) Time taken to get recommendations: 55 ms, Memory increase: 1731 KB.
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:9 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:9 [found 46 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1650 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:10 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:10 [found 22 items] (2ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 2 ms, Memory increase: 1645 KB.
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:12 ...
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] completion at /home/cyjun0304/Desktop/debug_pylance/main.py:24:12 [found 7 items] (1ms)
(1460942) IntelliCode: current invocation did not produce any meaningful tokens.
(1460942) Time taken to get recommendations: 1 ms, Memory increase: 1643 KB.
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [FG] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [FG] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
[Info - 7:18:26 AM] (1460942) SourceFile: Received fs event 'change' for path '/home/cyjun0304/Desktop/debug_pylance/main.py'
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5017ms)
[Info - 7:18:26 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (5017ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5019ms)
[Info - 7:18:26 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (5019ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4801ms)
[Info - 7:18:31 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4801ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4802ms)
[Info - 7:18:31 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4802ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: getSemanticTokens delta
(1460942) [BG(1)] getSemanticTokens delta previousResultId:1676877475596 at /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] getSemanticTokens delta previousResultId:1676877475596 at /home/cyjun0304/Desktop/debug_pylance/main.py (4666ms)
[Info - 7:18:35 AM] (1460942) [BG(1)] Long operation: getSemanticTokens delta previousResultId:1676877475596 at /home/cyjun0304/Desktop/debug_pylance/main.py (4666ms)
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4863ms)
[Info - 7:18:40 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4863ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4865ms)
[Info - 7:18:40 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4865ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4808ms)
[Info - 7:18:45 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4808ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4809ms)
[Info - 7:18:45 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4809ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: resumeAnalysis
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4819ms)
[Info - 7:18:50 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4819ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4821ms)
[Info - 7:18:50 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4821ms)
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4760ms)
[Info - 7:18:55 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4760ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4763ms)
[Info - 7:18:55 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4763ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4770ms)
[Info - 7:18:59 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4770ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4772ms)
[Info - 7:18:59 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4772ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (1ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4853ms)
[Info - 7:19:04 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4853ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4855ms)
[Info - 7:19:04 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4855ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: setFileOpened
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py ...
(1460942) [BG(1)] parsing: /home/cyjun0304/Desktop/debug_pylance/main.py (2ms)
(1460942) [BG(1)] binding: /home/cyjun0304/Desktop/debug_pylance/main.py (0ms)
(1460942) [BG(1)] checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4767ms)
[Info - 7:19:09 AM] (1460942) [BG(1)] Long operation: checking: /home/cyjun0304/Desktop/debug_pylance/main.py (4767ms)
(1460942) [BG(1)] analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4769ms)
[Info - 7:19:09 AM] (1460942) [BG(1)] Long operation: analyzing: /home/cyjun0304/Desktop/debug_pylance/main.py (4769ms)
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: markFilesDirty
(1460942) Background analysis message: analyze
(1460942) [BG(1)] indexing: /home/cyjun0304/Desktop/debug_pylance/main.py [found 7] (0ms)
(1460942) Indexing Done: /home/cyjun0304/Desktop/debug_pylance/main.py
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getSemanticTokens delta
(1460942) [BG(1)] getSemanticTokens delta previousResultId:1676877511137 at /home/cyjun0304/Desktop/debug_pylance/main.py (4ms)
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: resumeAnalysis
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
(1460942) Background analysis message: getDiagnosticsForRange
Is this a duplicate of https://github.com/microsoft/pylance-release/issues/3908?
can you set this "python.analysis.useLibraryCodeForTypes: false" and see whether it improves performance?
After I set "python.analysis.useLibraryCodeForTypes: false", the performance became normal. However, I cannot use type hint and parameter hint for my own libraries in other workspaces.
ya, we still need to figure out which part of sklearn is causing perf issues.
see how to generate some local stubs for sklearn https://github.com/microsoft/pyright/blob/main/docs/type-stubs.md#generating-type-stubs
@wangtz0607 I don't believe it is the same issue. your case is just parsing/binding taking time when a module is used the first time. once those are done, it is fast second time and after. if that is not the case, please open a new issue with repro steps and logs as the issue creator did.
@erictraut can you take a look? it looks like some of expressions are taking long time to analyze. I modified code a bit to see logs and for some expressions, getFunctionInferredReturnType and isAfterNodeReachable are taking a few seconds. can we adjust checkCodeFlowTooComplex code to consider these cases so we can skip some analysis for perf?
(17416) Background analysis message: analyze
(17416) [BG(1)] runWithCancellationToken ...
(17416) [BG(1)] analyzing: c:\pyTemp\junk\lib\test.py ...
(17416) [BG(1)] parsing: c:\pyTemp\junk\lib\test.py (1ms)
(17416) [BG(1)] binding: c:\pyTemp\junk\lib\test.py (1ms)
(17416) [BG(1)] checking: c:\pyTemp\junk\lib\test.py ...
(17416) [BG(1)] evaluateTypesForStatement ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] isAfterNodeReachable ...
(17416) [BG(1)] isAfterNodeReachable ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] isAfterNodeReachable ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] evaluateTypeForSubnode ...
(17416) [BG(1)] evaluateTypesForStatement ["arr" (scipy.sparse._sputils) [183:9]] (336ms)
(17416) [BG(1)] evaluateTypeForSubnode ["arr" (scipy.sparse._sputils) [183:9]] (336ms)
(17416) [BG(1)] getTypeOfExpression ["arr.max" (scipy.sparse._sputils) [190:30]] (339ms)
(17416) [BG(1)] getTypeOfExpression ["arr.min" (scipy.sparse._sputils) [191:30]] (503ms)
(17416) [BG(1)] isAfterNodeReachable [function '"get_index_dtype" [147:5]' (scipy.sparse._sputils) [147:1]] (503ms)
(17416) [BG(1)] evaluateTypeForSubnode ...
(17416) [BG(1)] evaluateTypesForStatement ...
(17416) [BG(1)] getTypeOfExpression ["idx_dtype" (scipy.sparse._compressed) [1017:19]] (70ms)
(17416) [BG(1)] evaluateTypeForSubnode ...
(17416) [BG(1)] evaluateTypesForStatement ["nnzs" (scipy.sparse._compressed) [1016:9]] (67ms)
(17416) [BG(1)] evaluateTypeForSubnode ["nnzs" (scipy.sparse._compressed) [1016:9]] (67ms)
(17416) [BG(1)] evaluateTypesForStatement ["self.indices" (scipy.sparse._compressed) [1198:13]] (252ms)
(17416) [BG(1)] evaluateTypeForSubnode ["self.indices" (scipy.sparse._compressed) [1198:13]] (252ms)
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] isAfterNodeReachable ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ["M.nonzero" (scipy.sparse._coo) [190:38]] (135ms)
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] evaluateTypeForSubnode ...
(17416) [BG(1)] evaluateTypesForStatement ["summed" (scipy.sparse._coo) [528:9]] (222ms)
(17416) [BG(1)] evaluateTypeForSubnode ["summed" (scipy.sparse._coo) [528:9]] (222ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'to_native' (scipy.sparse._sputils)] (653ms)
(17416) [BG(1)] getTypeOfExpression ["mask.all" (scipy.sparse._coo) [311:20]] (2046ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (2046ms)
(17416) [BG(1)] getTypeOfExpression ["self.data.astype" (scipy.sparse._coo) [195:25]] (2281ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (2281ms)
(17416) [BG(1)] getTypeOfExpression ["self._check" (scipy.sparse._coo) [197:9]] (2313ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (2313ms)
(17416) [BG(1)] isAfterNodeReachable [function '"__init__" [127:9]' (scipy.sparse._coo) [127:5]] (2313ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: isAfterNodeReachable (2313ms)
(17416) [BG(1)] getTypeOfExpression ["self._set_self" (scipy.sparse._compressed) [55:21]] (2314ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (2314ms)
(17416) [BG(1)] getTypeOfExpression ["self.indices.max" (scipy.sparse._compressed) [96:33]] (3340ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (3340ms)
(17416) [BG(1)] getTypeOfExpression ["self._swap" (scipy.sparse._compressed) [100:47]] (3341ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (3341ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'to_native' (scipy.sparse._sputils)] (184ms)
(17416) [BG(1)] getTypeOfExpression ["self.data.astype" (scipy.sparse._compressed) [104:25]] (3552ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (3552ms)
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] evaluateTypeForSubnode ...
(17416) [BG(1)] evaluateTypesForStatement ["self.indices" (scipy.sparse._compressed) [159:9]] (91ms)
(17416) [BG(1)] evaluateTypeForSubnode ["self.indices" (scipy.sparse._compressed) [159:9]] (91ms)
(17416) [BG(1)] getTypeOfExpression ["self.indices.max" (scipy.sparse._compressed) [186:20]] (92ms)
(17416) [BG(1)] getTypeOfExpression ["self.indices.min" (scipy.sparse._compressed) [189:20]] (92ms)
(17416) [BG(1)] getTypeOfExpression ["np.diff(self.indptr).min" (scipy.sparse._compressed) [192:20]] (105ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'check_format' (scipy.sparse._compressed)] (105ms)
(17416) [BG(1)] getTypeOfExpression ["self.check_format" (scipy.sparse._compressed) [106:9]] (3657ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (3657ms)
(17416) [BG(1)] isAfterNodeReachable [function '"__init__" [26:9]' (scipy.sparse._compressed) [26:5]] (3657ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: isAfterNodeReachable (3657ms)
(17416) [BG(1)] isAfterNodeReachable [function '"_joint_probabilities_nn" [70:5]' (sklearn.manifold._t_sne) [70:1]] (3661ms)
[Info - 3:27:03 PM] (17416) [BG(1)] Long operation: isAfterNodeReachable (3661ms)
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'check_pairwise_arrays' (sklearn.metrics.pairwise)] (1497ms)
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] isAfterNodeReachable ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] _getFunctionInferredReturnType ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ...
(17416) [BG(1)] getTypeOfExpression ["self.dispatch_one_batch" (joblib.parallel) [1085:16]] (62ms)
(17416) [BG(1)] getTypeOfExpression ["self.dispatch_one_batch" (joblib.parallel) [1088:19]] (62ms)
(17416) [BG(1)] getTypeOfExpression ["self._backend.retrieval_contex <shortened> " (joblib.parallel) [1097:18]] (62ms)
(17416) [BG(1)] getTypeOfExpression ["self._print" (joblib.parallel) [1101:13]] (63ms)
(17416) [BG(1)] getTypeOfExpression ["self._backend.stop_call" (joblib.parallel) [1106:17]] (63ms)
(17416) [BG(1)] getTypeOfExpression ["self._terminate_backend" (joblib.parallel) [1108:17]] (65ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function '__call__' (joblib.parallel)] (65ms)
(17416) [BG(1)] getTypeOfExpression ["super().__call__" (sklearn.utils.parallel) [63:16]] (65ms)
(17416) [BG(1)] isAfterNodeReachable [function '"__call__" [40:9]' (sklearn.utils.parallel) [40:5]] (66ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function '_parallel_pairwise' (sklearn.metrics.pairwise)] (72ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'pairwise_distances' (sklearn.metrics.pairwise)] (1584ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'pairwise_distances_chunked' (sklearn.metrics.pairwise)] (1595ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'kneighbors' (sklearn.neighbors._base)] (1789ms)
(17416) [BG(1)] getTypeOfExpression ["A_ind.ravel" (sklearn.neighbors._base) [1003:22]] (1792ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'kneighbors_graph' (sklearn.neighbors._base)] (1843ms)
(17416) [BG(1)] getTypeOfExpression ["knn.kneighbors_graph" (sklearn.manifold._t_sne) [963:28]] (1874ms)
(17416) [BG(1)] getTypeOfExpression ["self._tsne" (sklearn.manifold._t_sne) [1012:16]] (5611ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (5611ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function '_fit' (sklearn.manifold._t_sne)] (5637ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: _getFunctionInferredReturnType (5637ms)
(17416) [BG(1)] getTypeOfExpression ["self._fit" (sklearn.manifold._t_sne) [1119:21]] (5646ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: getTypeOfExpression (5646ms)
(17416) [BG(1)] _getFunctionInferredReturnType [Function 'fit_transform' (sklearn.manifold._t_sne)] (5646ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: _getFunctionInferredReturnType (5646ms)
(17416) [BG(1)] evaluateTypesForStatement [Assignment (lib.test) [12:9]] (5646ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: evaluateTypesForStatement (5646ms)
(17416) [BG(1)] checking: c:\pyTemp\junk\lib\test.py (5663ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: checking: c:\pyTemp\junk\lib\test.py (5663ms)
(17416) [BG(1)] analyzing: c:\pyTemp\junk\lib\test.py (5665ms)
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: analyzing: c:\pyTemp\junk\lib\test.py (5665ms)
(17416) [BG(1)] runWithCancellationToken (5665ms) [f:0, t:1, p:0, i:0, b:1]
[Info - 3:27:05 PM] (17416) [BG(1)] Long operation: runWithCancellationToken (5665ms)
this log is from this option
"python.analysis.logLevel": "Trace",
"python.analysis.logTypeEvaluationTime": true,
"python.analysis.minimumLoggingThreshold": 500,
with some custom logging to get getFunctionInferredReturnType log.
@heejaechang, can you provide a self-contained minimal repro in text form?
@erictraut the issue creator already provided one.
https://github.com/microsoft/pylance-release/issues/3978#issue-1591298656
import numpy as np
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA
class sklearn_test(object):
def process(self, reduction_method = 'pca'):
if reduction_method == 'pca' :
self.reduction_method = PCA(n_components=3, random_state=2023)
elif reduction_method == 'tsne' :
self.reduction_method = TSNE(n_components=3, random_state=2023)
value = np.random.random((10,10))
reduced_values = self.reduction_method.fit_transform(value)
return reduced_values
if __name__ == "__main__":
a = sklearn_test()
reduced_values = a.process()
print(reduced_values)
b = dict()
for key, value in b.items():
print("hi")
once you have scikit-learn installed. just paste the code above so our checker runs.
from the log, I can reduce it to this.
from sklearn.manifold import TSNE
method = TSNE(n_components=3, random_state=2023)
reduced_values = method.fit_transform(None)
since the log says tsne's fit_transform is the one that shows long operation.
I've spent some time looking at this issue. There is nothing that stands out as unexpected other than the fact that sklearn is completely untyped and has very deep call chains that often invoke functionality in scipy, which is likewise completely untyped. No individual function in these call chains are all that complex, but analysis of each requires the type analyzer to understand the return type of each of the functions that it invokes (hence the need to call getFunctionInferredReturnType internally to infer the return type of each called function.
Pylance already ships partial stubs for both sklearn and scipy. I recommend augmenting these stubs with a few more classes and methods. You don't need to add type information for all modules or symbols. Just a few strategic ones should make a big difference to the experience here.
Longer term, the right answer is to convince the maintainers of sklearn and scipy to add proper inlined type information. Many library authors are doing this already. If they cannot be convinced, then it might make sense to spearhead an effort to add third-party stubs to typeshed for these libraries.
@gramster has been working on scikit-learn stubs. assigning to Graham to experiment with his stubs.
according to logs from 3 sciki-learn issues. one common source of perf issues is scipy.sparse. if we can put partial stubs for modules under that sub package, we could have at least 50% perf win.
I'm not seeing this slow performance in pylance v2024.3.1, but maybe i had a newer version of sklearn
