Use `importlib.metadata` when trying to get package version

[dev-jam]

Issue Report Checklist

• [X ] Searched the issues page for similar reports
• [x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• [ ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• [ ] Could not reproduce inside jupyter qtconsole (if console-related)
• [ ] Tried basic troubleshooting (if a bug/error)
  • [ ] Restarted Spyder
  • [ ] Reset preferences with spyder --reset
  • [ ] Reinstalled the latest version of Anaconda
  • [ ] Tried the other applicable steps from the Troubleshooting Guide
• [ ] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Spyder 5.4.2 shows it is missing the dependency 'pylsp_black' on startup.

I checked and debugged spyder and found a small bug in the 'get_module_version' function in utils/programs.py. This bug is also present in the latest trunk code. When checking dependencies, first the 'module name' is checked for a version with 'getattr'. 'pylsp_black' gives 'none' here. Then the 'get_package_version' function is called with the 'module_name' instead of the 'package_name':

get_package_version(module_name)

This 'get_package_version' function needs the 'package_name' and not the 'module_name'.

So spyder checks the version by:

ver = pkg_resources.get_distribution('pylsp_black').version

but it needs to be:

ver = pkg_resources.get_distribution('python-lsp-black').version

When I use the 'package_name' / 'python-lsp-black' instead of the 'module_name' / 'pylsp_black' through the console, I get a correct version number.

What steps reproduce the problem?

What is the expected output? What do you see instead?

Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

• Spyder version:
• Python version:
• Qt version:
• PyQt version:
• Operating System name/version:

Dependencies

PASTE DEPENDENCIES HERE

[dalthviz]

Hi @dev-jam thank you for the feedback! About your findings:

I checked and debugged spyder and found a small bug in the 'get_module_version' function in utils/programs.py

Could you point out the specific place where that function is used to give the logic a check, please?

Also, could it be possible for you to check if you are able to reproduce this after updating to Spyder 5.5.0? I'm running that version on Windows and I'm able to see the python-lsp-black version without issue. I'm not getting the missing dependecy dialog and I'm able to open the dependencies dialog to see the package version - Help > Dependencies...:

So understanding why is happening this for you on Spyder 5.4.2 but not on Spyder 5.5.0 for me could be nice 🤔

Any other info to better understand the issue and your findings for a possible fix are greatly appreciated. Let us know!

[dev-jam]

Hi @dalthviz,

Could you point out the specific place where that function is used to give the logic a check, please?

In Spyder 5.4.2, utils/program.py, line 830, the function 'get_module_version': with pylsp_black, 'ver' has value 'None' and then it tries 'get_package_version' which tries to get the version from pkg_resources:

ver = pkg_resources.get_distribution(package_name).version

But this last function needs the package_name and not the module_name.

I installed spyder 5.5.0 through pipx and it does not have the dependency problem anymore. The problematic code is still in 5.5.0 but I don't think this code is used anymore, see the comment in utils/programs.py from spyder 5.4.2 line 852:

# When support for Python 3.7 and below is dropped, this can be replaced
# with the built-in importlib.metadata.version

So I think 5.5.0 uses the new built-in importlib.metadata.version successfully and never gets to that function anymore.

I think most dependencies are successfully found with the 'get_module_version' function. And maybe those few modules that did not return a version in the past might have had a package name that was equal to the module name. But that is a guess.

I don't think a fix is really relevant for me because I use debian stable and those bugs are never fixed in there. So i will stay with the pipx spyder 5.5.0. Maybe other people are interested but at least the bug is documented now.

Thanks for your help.

[dalthviz]

Oh I see, thank you for the new info @dev-jam ! While checking the lines I noticed that the same message and code from 5.4.2 is indeed still present even in the 5.x branch 🤔

https://github.com/spyder-ide/spyder/blob/739457e0006df05b347de75916fe3714a4a1a309/spyder/utils/programs.py#L846-L855

But nice to hear that you were able to use Spyder without the missing dependency dialog by installing 5.5.0!

Just in case, pinging @ccordoba12 . Seems like we should do a change to address the comment above since Spyder 5.5.0 dropped Python 3.7 support, right?

[ccordoba12]

Just in case, pinging @ccordoba12 . Seems like we should do a change to address the comment above since Spyder 5.5.0 dropped Python 3.7 support, right?

Yep, that's right. I'm going to tag this for beta1, but please try to address it for alpha3 if you have time.

[dev-jam]

I got an update on the bug. I was running 5.5.0 through pipx but today I got an update from proposed updates (5.4.2+ds-5+deb12u1) in debian stable with changelog:

• Fix interface language auto-configuration (closes: #1054475)

So I tried this new 5.4.2, you never know, and the dependency bug was gone. To investigate further, I uninstalled the pipx version. Same result, no bug. Then I downgraded back to the version in stable and still the bug is gone. At the moment I cannot reproduce the bug anymore. It seems that or running 5.5.0 changed something in my user settings or upgrading to 5.4.2+ds-5+deb12u1 changed something irreversible in my system. I still have one debian machine at work with the bug. Gonna check it out if I can find something there.

When testing this bug, I did try to delete all the user stuff but maybe I missed something. Do you happen to know in which folders spyder has conf files? For testing I deleted:

~/.config/spyder-py3 ~/.cache/Spyder ~/.local/share/Spyder

[dalthviz]

Thank you for the new info @dev-jam !

When testing this bug, I did try to delete all the user stuff but maybe I missed something. Do you happen to know in which folders spyder has conf files?

All the conf files I think are at <user home dir>/.spyder-py3. You can also run spyder --reset from the terminal to reset all preferences (that will also show a list of all the files related with Spyder preferences/config that the --reset command removes)

[mrclary]

I have a user at my work reporting the same error running our Windows standalone application version 5.5.1. The application runs without issue after install and after several restarts, but regresses after an indeterminate time (maybe after a computer reboot).

I have been unable to reproduce the issue on my Windows VM, but I will investigate further.

[mrclary]

Upon further investigating my colleague's issue I discovered that there were 2 dist-info directories in the pkgs directory: C:\Program Files\Spyder\pkgs\python-lsp-black-1.3.0.dist-info C:\Program Files\Spyder\pkgs\python-lsp-black-2.0.0.dist-info

Both pkg_resources.get_distribution('python-lsp-black').version and importlib.metadata.version('python-lsp-black') returned 1.3.0 instead of 2.0.0, in Spyder's internal console. So, Spyder is identifying the wrong version due to the presence of this superflous dist-info directory.

We don't know how this situation came about, but I suspect that when updating Spyder somehow some old directories/files were not completely removed and the new installation just installed over them 🤷‍♂️. We've completely uninstalled all versions of Spyder, ensured all program files and directories were completely removed, and reinstalled v5.5.1. Initially this is working, but we will continue to monitor the situation and report back if there are new developments.