pipdeptree
pipdeptree copied to clipboard
Published
20 hours ago •
tox-dev
tox-dev
`pipdeptree -u` crashes with "TypeError: expected string or bytes-like object"
Tested with git master.
Versions:
$ python ./pipdeptree.py -v
2.0.0b1
$ python -V
Python 3.8.6
$ pip -V
pip 20.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)
Error message:
Traceback (most recent call last):
File "/home/luxu/.local/bin/pipdeptree", line 8, in <module>
sys.exit(main())
File "/home/luxu/.local/lib/python3.8/site-packages/pipdeptree.py", line 596,
in main
conflicting = conflicting_deps(tree)
File "/home/luxu/.local/lib/python3.8/site-packages/pipdeptree.py", line 476,
in conflicting_deps
if req.is_conflicting():
File "/home/luxu/.local/lib/python3.8/site-packages/pipdeptree.py", line 264,
in is_conflicting
return self.installed_version not in req_obj
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3125,
in __contains__
return self.specifier.contains(item, prereleases=True)
File "/usr/lib/python3.8/site-packages/packaging/specifiers.py", line 789, in
contains
item = parse(item)
File "/usr/lib/python3.8/site-packages/packaging/version.py", line 56, in parse
return Version(version)
File "/usr/lib/python3.8/site-packages/packaging/version.py", line 275, in __init__
match = self._regex.search(version)
TypeError: expected string or bytes-like object
Python modules installed in user home:
$ pip list --user
Package Version
----------------- ----------
ads 0.12.3
astrocats 0.3.37
astropy 4.2
attrdict 2.0.1
babelfish 0.5.5
bashlex 0.15
bencoder.pyx 2.0.1
blinker 1.4
blist 1.3.6
bokeh 2.2.3
certifi 2020.11.8
compiledb 0.10.1
corner 2.1.0
Cython 0.29.21
dcnnt 0.3.6
dogpile.cache 1.1.1
dynesty 1.0.1
emcee 3.0.2
enzyme 0.4.1
extinction 0.4.5
feedparser 6.0.2
flatbencode 0.2.1
gitdb 4.0.5
gitdb2 4.0.2
GitPython 3.1.11
grbpy 0.3.0
guessit 3.1.1
html2text 2020.1.16
httpretty 1.0.3
inflect 5.0.2
ipython-genutils 0.2.0
jsonschema 3.2.0
jupyter-core 4.7.0
mock 4.0.2
mosfit 1.1.7
mpi4py 3.0.3
nbformat 5.0.8
nbstripout 0.3.9
numexpr 2.7.1
orderedattrdict 1.6.0
palettable 3.3.0
pandas 1.1.4
panwid 0.2.5
pbr 5.5.1
pipdeptree 2.0.0b1
pycryptodome 3.9.9
pyerfa 1.7.1.1
pyrsistent 0.17.3
pysrt 1.1.2
qbittorrent-api 2020.11.12
qbittorrentui 0.1.3
qr-filetransfer 2.7
qrcode 6.1
raccoon 3.0.0
rarfile 4.0
rebulk 2.0.1
schwimmbad 0.3.1
seaborn 0.11.0
selenium 3.141.0
setuptools-scm 4.1.2
sgmllib3k 1.0.0
shutilwhich 1.1.0
smmap 3.0.4
smmap2 3.0.1
stevedore 3.2.2
subliminal 2.1.0
tabulate 0.8.7
torf 3.1.3
tornado 6.1
tqdm 4.53.0
traitlets 5.0.5
transmissionrpc 0.11
typing-extensions 3.7.4.3
urwid 2.1.2
urwid-utils 0.1.2
Werkzeug 1.0.1
wheel 0.35.1
Thanks for reporting this. I tried recreating an environment with the same versions of python, pip and the packages that you have mentioned, but I faced 2 problems:
- Couldn't install
grbpy==0.3.0. Is it a private package? - Building
mpi4pytakes forever (VM running ubuntu bionic)
To help me debug this further, can you please apply the following diff on top of latest master and send me the output of pipdeptree on the same env?
diff --git a/pipdeptree.py b/pipdeptree.py
index eccab33..e3c814a 100644
--- a/pipdeptree.py
+++ b/pipdeptree.py
@@ -213,7 +213,11 @@ class ReqPackage(Package):
ver_spec = (self.version_spec if self.version_spec else '')
req_version_str = '{0}{1}'.format(self.project_name, ver_spec)
req_obj = pkg_resources.Requirement.parse(req_version_str)
- return self.installed_version not in req_obj
+ try:
+ return self.installed_version not in req_obj
+ except Exception:
+ print(self.dist.as_dict())
+ return False
def render_as_root(self, frozen):
if not frozen:
The error message is now as follows, there is another error with the new debug code
Traceback (most recent call last):
File "./pipdeptree.py", line 217, in is_conflicting
return self.installed_version not in req_obj
File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3125,
in __contains__
return self.specifier.contains(item, prereleases=True)
File "/usr/lib/python3.8/site-packages/packaging/specifiers.py", line 789, in
contains
item = parse(item)
File "/usr/lib/python3.8/site-packages/packaging/version.py", line 56, in parse
return Version(version)
File "/usr/lib/python3.8/site-packages/packaging/version.py", line 275, in __init__
match = self._regex.search(version)
TypeError: expected string or bytes-like object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./pipdeptree.py", line 878, in <module>
sys.exit(main())
File "./pipdeptree.py", line 838, in main
conflicts = conflicting_deps(tree)
File "./pipdeptree.py", line 661, in conflicting_deps
if req.is_conflicting():
File "./pipdeptree.py", line 219, in is_conflicting
print(self.dist.as_dict())
AttributeError: 'NoneType' object has no attribute 'as_dict'
I tried on another machine with the same versions of python, pip and patched master version of pipdeptree, got the same error. The modules installed there are (there are not grbpy and mpi4py)
Edit: use freeze format
$ pip list --user --format=freeze
aiohttp==3.7.3
async-timeout==3.0.1
attrdict==2.0.1
babelfish==0.5.5
bencoder.pyx==2.0.1
blinker==1.4
blist==1.3.6
bpytop==1.0.50
dcnnt==0.3.6
dogpile.cache==1.1.1
emcee==3.0.2
enum34==1.1.10
enzyme==0.4.1
evdev==1.3.0
flatbencode==0.2.1
guessit==3.1.1
html2text==2020.1.16
Js2Py==0.70
keyboard==0.13.5
mock==4.0.2
multidict==5.0.2
natsort==7.1.0
orderedattrdict==1.6.0
panwid==0.2.5
pbr==5.5.1
pip-autoremove==0.9.1
pipdeptree==1.0.0
py-mini-racer==0.4.0
PyExecJS==1.5.1
pyjsparser==2.7.1
pynput==1.7.1
pysrt==1.1.2
python-dateutil==2.8.1
pyTwistyScrambler==1.2
pytz==2020.4
qbittorrent-api==2020.11.12
qbittorrentui==0.1.3
raccoon==3.0.0
rarfile==4.0
rebulk==2.0.1
spark-parser==1.8.9
stevedore==3.2.2
stig==0.11.2a0
subliminal==2.1.0
tabulate==0.8.7
torf==3.1.3
typing-extensions==3.7.4.3
tzlocal==2.1
urwid==2.1.2
urwid-utils==0.1.2
urwidtrees==1.0.3
xdis==5.0.5
xmltodict==0.12.0
yapf==0.30.0
yarl==1.6.3
I tried with the updated list and couldn't reproduce the error. Can you please try the following patch?
diff --git a/pipdeptree.py b/pipdeptree.py
index eccab33..29b702c 100644
--- a/pipdeptree.py
+++ b/pipdeptree.py
@@ -213,7 +213,11 @@ class ReqPackage(Package):
ver_spec = (self.version_spec if self.version_spec else '')
req_version_str = '{0}{1}'.format(self.project_name, ver_spec)
req_obj = pkg_resources.Requirement.parse(req_version_str)
- return self.installed_version not in req_obj
+ try:
+ return self.installed_version not in req_obj
+ except Exception:
+ print(self.as_dict())
+ return False
def render_as_root(self, frozen):
if not frozen:
There is dependency list printed with the try/catch. The result is
$ python ./pipdeptree.py -u
{'key': 'requests', 'package_name': 'requests', 'installed_version': <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>, 'required_version': '>=2.0'}
{'key': 'requests', 'package_name': 'requests', 'installed_version': <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>, 'required_version': '>=2.16.0'}
Warning!!! Possibly conflicting dependencies found:
* aiohttp==3.7.3
- attrs [required: >=17.3.0, installed: ?]
* dcnnt==0.3.6
- pycryptodome [required: >=3.9.3, installed: ?]
* panwid==0.2.5
- raccoon [required: ==2.1.5, installed: 3.0.0]
* pynput==1.7.1
- python-xlib [required: >=0.17, installed: ?]
* qbittorrentui==0.1.3
- urwid [required: ==2.0.1, installed: 2.1.2]
* stig==0.11.2a0
- pyxdg [required: Any, installed: ?]
* subliminal==2.1.0
- beautifulsoup4 [required: >=4.4.0, installed: ?]
------------------------------------------------------------------------
bencoder.pyx==2.0.1
bpytop==1.0.50
- psutil [required: >=5.7.0,<6.0.0, installed: 5.7.3]
dcnnt==0.3.6
- pycryptodome [required: >=3.9.3, installed: ?]
emcee==3.0.2
- numpy [required: Any, installed: 1.19.4]
enum34==1.1.10
html2text==2020.1.16
Js2Py==0.70
- pyjsparser [required: >=2.5.1, installed: 2.7.1]
- six [required: >=1.10, installed: 1.15.0]
- tzlocal [required: >=1.2, installed: 2.1]
- pytz [required: Any, installed: 2020.4]
keyboard==0.13.5
pip-autoremove==0.9.1
pipdeptree==1.0.0
- pip [required: >=6.0.0, installed: 20.2]
py-mini-racer==0.4.0
pynput==1.7.1
- evdev [required: >=1.3, installed: 1.3.0]
- python-xlib [required: >=0.17, installed: ?]
- six [required: Any, installed: 1.15.0]
pyTwistyScrambler==1.2
- appdirs [required: Any, installed: 1.4.4]
- packaging [required: Any, installed: 20.4]
- PyExecJS [required: Any, installed: 1.5.1]
- six [required: >=1.10.0, installed: 1.15.0]
- pyparsing [required: Any, installed: 2.4.7]
- six [required: Any, installed: 1.15.0]
qbittorrentui==0.1.3
- attrdict [required: >=2.0.0,<=2.0.1, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- blinker [required: ==1.4, installed: 1.4]
- panwid [required: ==0.2.5, installed: 0.2.5]
- blist [required: Any, installed: 1.3.6]
- orderedattrdict [required: Any, installed: 1.6.0]
- raccoon [required: ==2.1.5, installed: 3.0.0]
- tabulate [required: Any, installed: 0.8.7]
- six [required: Any, installed: 1.15.0]
- urwid [required: Any, installed: 2.1.2]
- urwid-utils [required: >=0.1.2, installed: 0.1.2]
- urwid [required: Any, installed: 2.1.2]
- qbittorrent-api [required: Any, installed: 2020.11.12]
- attrdict [required: >=2.0.0, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- requests [required: >=2.16.0, installed: <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>]
- six [required: Any, installed: 1.15.0]
- urllib3 [required: >=1.24.2, installed: 1.25.10]
- urwid [required: ==2.0.1, installed: 2.1.2]
spark-parser==1.8.9
- click [required: Any, installed: 7.1.2]
stig==0.11.2a0
- aiohttp [required: >=3,<4, installed: 3.7.3]
- async-timeout [required: >=3.0,<4.0, installed: 3.0.1]
- attrs [required: >=17.3.0, installed: ?]
- chardet [required: >=2.0,<4.0, installed: 3.0.4]
- multidict [required: >=4.5,<7.0, installed: 5.0.2]
- typing-extensions [required: >=3.6.5, installed: 3.7.4.3]
- yarl [required: >=1.0,<2.0, installed: 1.6.3]
- idna [required: >=2.0, installed: 2.10]
- multidict [required: >=4.0, installed: 5.0.2]
- async-timeout [required: Any, installed: 3.0.1]
- blinker [required: Any, installed: 1.4]
- natsort [required: Any, installed: 7.1.0]
- pyxdg [required: Any, installed: ?]
- urwid [required: >=2.0, installed: 2.1.2]
- urwidtrees [required: >=1.0.3dev0, installed: 1.0.3]
- mock [required: Any, installed: 4.0.2]
- urwid [required: >=1.1.0, installed: 2.1.2]
subliminal==2.1.0
- appdirs [required: >=1.3, installed: 1.4.4]
- babelfish [required: >=0.5.2, installed: 0.5.5]
- beautifulsoup4 [required: >=4.4.0, installed: ?]
- chardet [required: >=2.3.0, installed: 3.0.4]
- click [required: >=4.0, installed: 7.1.2]
- dogpile.cache [required: >=0.6.0, installed: 1.1.1]
- decorator [required: >=4.0.0, installed: 4.4.2]
- stevedore [required: >=3.0.0, installed: 3.2.2]
- pbr [required: >=2.0.0,!=2.1.0, installed: 5.5.1]
- enzyme [required: >=0.4.1, installed: 0.4.1]
- guessit [required: >=3.0.0, installed: 3.1.1]
- babelfish [required: Any, installed: 0.5.5]
- python-dateutil [required: Any, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- rebulk [required: ==2.*, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- six [required: Any, installed: 1.15.0]
- pysrt [required: >=1.0.1, installed: 1.1.2]
- chardet [required: Any, installed: 3.0.4]
- pytz [required: >=2012c, installed: 2020.4]
- rarfile [required: >=2.7, installed: 4.0]
- requests [required: >=2.0, installed: <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>]
- six [required: >=1.9.0, installed: 1.15.0]
- stevedore [required: >=1.20.0, installed: 3.2.2]
- pbr [required: >=2.0.0,!=2.1.0, installed: 5.5.1]
torf==3.1.3
- flatbencode [required: ==0.2.*, installed: 0.2.1]
xdis==5.0.5
xmltodict==0.12.0
yapf==0.30.0
Thanks. I think I understand the problem now, but still not able to reproduce it at my end.
So you have 2 packages qbittorrent-api and subliminal which depend on requests. Looks like it's using the requests module which is packaged with pip under pip._vendor.requests instead of a standalone requests module installed using pip. In your previous reply, you have mentioned the output of pip list --user --format=freeze which also doesn't list requests.
Strangely, the pipdeptree.guess_version('requests') call seems to be returning the module object /usr/lib/python3.8/site-packages/requests/__version__.py instead of string.
To reproduce it, I created a virtualenv with the above mentioned packages and then uninstalled requests. It doesn't fail for me but I get the following warning:
Warning!!! Possibly conflicting dependencies found:
* qbittorrent-api==2020.11.12
- requests [required: >=2.16.0, installed: ?]
* subliminal==2.1.0
- requests [required: >=2.0, installed: ?]
Sorry for the trouble, but can you please try one final patch and give me the output,
diff --git a/pipdeptree.py b/pipdeptree.py
index eccab33..9e57500 100644
--- a/pipdeptree.py
+++ b/pipdeptree.py
@@ -66,6 +66,9 @@ def guess_version(pkg_key, default='?'):
"""
try:
m = import_module(pkg_key)
+ print(pkg_key)
+ print(m)
+ print(type(m))
except ImportError:
return default
else:
To get it working, you can just try installing requests>=2.16.0 explicitly. I think that will fix it.
For the request module I'm using: I am using Arch Linux and have python-requests (version 2.24.0) from official repo installed. If there is nothing wrong with Arch's packaging, I hope pipdeptree could work in cases like mine since I don't (need to) keep two copies/versions of a module in my system. If that's easy to do, of course.
To get it working, you can just try installing requests>=2.16.0 explicitly. I think that will fix it.
Yes, it worked after pip install --user requests==2.xx.0 with 16<=xx<24 (I guess pip wouldn't install with the same version of requests already installed by system package manager?).
The output with the last patch (and the previous one also) is quite long:
idna
<module 'idna' from '/usr/lib/python3.8/site-packages/idna/__init__.py'>
<class 'module'>
idna
<module 'idna' from '/usr/lib/python3.8/site-packages/idna/__init__.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
{'key': 'requests', 'package_name': 'requests', 'installed_version': <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>, 'required_version': '>=2.0'}
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
urllib3
<module 'urllib3' from '/usr/lib/python3.8/site-packages/urllib3/__init__.py'>
<class 'module'>
urllib3
<module 'urllib3' from '/usr/lib/python3.8/site-packages/urllib3/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
{'key': 'requests', 'package_name': 'requests', 'installed_version': <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>, 'required_version': '>=2.16.0'}
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
pyparsing
<module 'pyparsing' from '/usr/lib/python3.8/site-packages/pyparsing.py'>
<class 'module'>
pyparsing
<module 'pyparsing' from '/usr/lib/python3.8/site-packages/pyparsing.py'>
<class 'module'>
packaging
<module 'packaging' from '/usr/lib/python3.8/site-packages/packaging/__init__.py'>
<class 'module'>
packaging
<module 'packaging' from '/usr/lib/python3.8/site-packages/packaging/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
pip
<module 'pip' from '/usr/lib/python3.8/site-packages/pip/__init__.py'>
<class 'module'>
pip
<module 'pip' from '/usr/lib/python3.8/site-packages/pip/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
numpy
<module 'numpy' from '/usr/lib/python3.8/site-packages/numpy/__init__.py'>
<class 'module'>
numpy
<module 'numpy' from '/usr/lib/python3.8/site-packages/numpy/__init__.py'>
<class 'module'>
decorator
<module 'decorator' from '/usr/lib/python3.8/site-packages/decorator.py'>
<class 'module'>
decorator
<module 'decorator' from '/usr/lib/python3.8/site-packages/decorator.py'>
<class 'module'>
psutil
<module 'psutil' from '/usr/lib/python3.8/site-packages/psutil/__init__.py'>
<class 'module'>
psutil
<module 'psutil' from '/usr/lib/python3.8/site-packages/psutil/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
psutil
<module 'psutil' from '/usr/lib/python3.8/site-packages/psutil/__init__.py'>
<class 'module'>
numpy
<module 'numpy' from '/usr/lib/python3.8/site-packages/numpy/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
pip
<module 'pip' from '/usr/lib/python3.8/site-packages/pip/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
packaging
<module 'packaging' from '/usr/lib/python3.8/site-packages/packaging/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
pyparsing
<module 'pyparsing' from '/usr/lib/python3.8/site-packages/pyparsing.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
urllib3
<module 'urllib3' from '/usr/lib/python3.8/site-packages/urllib3/__init__.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
idna
<module 'idna' from '/usr/lib/python3.8/site-packages/idna/__init__.py'>
<class 'module'>
appdirs
<module 'appdirs' from '/usr/lib/python3.8/site-packages/appdirs.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
click
<module 'click' from '/usr/lib/python3.8/site-packages/click/__init__.py'>
<class 'module'>
decorator
<module 'decorator' from '/usr/lib/python3.8/site-packages/decorator.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>Warning!!! Possibly conflicting dependencies found:
* aiohttp==3.7.3
- attrs [required: >=17.3.0, installed: ?]
* dcnnt==0.3.6
- pycryptodome [required: >=3.9.3, installed: ?]
* panwid==0.2.5
- raccoon [required: ==2.1.5, installed: 3.0.0]
* pynput==1.7.1
- python-xlib [required: >=0.17, installed: ?]
* qbittorrentui==0.1.3
- urwid [required: ==2.0.1, installed: 2.1.2]
* stig==0.11.2a0
- pyxdg [required: Any, installed: ?]
* subliminal==2.1.0
- beautifulsoup4 [required: >=4.4.0, installed: ?]
------------------------------------------------------------------------
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
chardet
<module 'chardet' from '/usr/lib/python3.8/site-packages/chardet/__init__.py'>
<class 'module'>
requests
<module 'requests' from '/usr/lib/python3.8/site-packages/requests/__init__.py'>
<class 'module'>
six
<module 'six' from '/usr/lib/python3.8/site-packages/six.py'>
<class 'module'>
bencoder.pyx==2.0.1
bpytop==1.0.50
- psutil [required: >=5.7.0,<6.0.0, installed: 5.7.3]
dcnnt==0.3.6
- pycryptodome [required: >=3.9.3, installed: ?]
emcee==3.0.2
- numpy [required: Any, installed: 1.19.4]
enum34==1.1.10
html2text==2020.1.16
Js2Py==0.70
- pyjsparser [required: >=2.5.1, installed: 2.7.1]
- six [required: >=1.10, installed: 1.15.0]
- tzlocal [required: >=1.2, installed: 2.1]
- pytz [required: Any, installed: 2020.4]
keyboard==0.13.5
pip-autoremove==0.9.1
pipdeptree==1.0.0
- pip [required: >=6.0.0, installed: 20.2]
py-mini-racer==0.4.0
pynput==1.7.1
- evdev [required: >=1.3, installed: 1.3.0]
- python-xlib [required: >=0.17, installed: ?]
- six [required: Any, installed: 1.15.0]
pyTwistyScrambler==1.2
- appdirs [required: Any, installed: 1.4.4]
- packaging [required: Any, installed: 20.4]
- PyExecJS [required: Any, installed: 1.5.1]
- six [required: >=1.10.0, installed: 1.15.0]
- pyparsing [required: Any, installed: 2.4.7]
- six [required: Any, installed: 1.15.0]
qbittorrentui==0.1.3
- attrdict [required: >=2.0.0,<=2.0.1, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- blinker [required: ==1.4, installed: 1.4]
- panwid [required: ==0.2.5, installed: 0.2.5]
- blist [required: Any, installed: 1.3.6]
- orderedattrdict [required: Any, installed: 1.6.0]
- raccoon [required: ==2.1.5, installed: 3.0.0]
- tabulate [required: Any, installed: 0.8.7]
- six [required: Any, installed: 1.15.0]
- urwid [required: Any, installed: 2.1.2]
- urwid-utils [required: >=0.1.2, installed: 0.1.2]
- urwid [required: Any, installed: 2.1.2]
- qbittorrent-api [required: Any, installed: 2020.11.12]
- attrdict [required: >=2.0.0, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- requests [required: >=2.16.0, installed: <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>]
- six [required: Any, installed: 1.15.0]
- urllib3 [required: >=1.24.2, installed: 1.25.10]
- urwid [required: ==2.0.1, installed: 2.1.2]
spark-parser==1.8.9
- click [required: Any, installed: 7.1.2]
stig==0.11.2a0
- aiohttp [required: >=3,<4, installed: 3.7.3]
- async-timeout [required: >=3.0,<4.0, installed: 3.0.1]
- attrs [required: >=17.3.0, installed: ?]
- chardet [required: >=2.0,<4.0, installed: 3.0.4]
- multidict [required: >=4.5,<7.0, installed: 5.0.2]
- typing-extensions [required: >=3.6.5, installed: 3.7.4.3]
- yarl [required: >=1.0,<2.0, installed: 1.6.3]
- idna [required: >=2.0, installed: 2.10]
- multidict [required: >=4.0, installed: 5.0.2]
- async-timeout [required: Any, installed: 3.0.1]
- blinker [required: Any, installed: 1.4]
- natsort [required: Any, installed: 7.1.0]
- pyxdg [required: Any, installed: ?]
- urwid [required: >=2.0, installed: 2.1.2]
- urwidtrees [required: >=1.0.3dev0, installed: 1.0.3]
- mock [required: Any, installed: 4.0.2]
- urwid [required: >=1.1.0, installed: 2.1.2]
subliminal==2.1.0
- appdirs [required: >=1.3, installed: 1.4.4]
- babelfish [required: >=0.5.2, installed: 0.5.5]
- beautifulsoup4 [required: >=4.4.0, installed: ?]
- chardet [required: >=2.3.0, installed: 3.0.4]
- click [required: >=4.0, installed: 7.1.2]
- dogpile.cache [required: >=0.6.0, installed: 1.1.1]
- decorator [required: >=4.0.0, installed: 4.4.2]
- stevedore [required: >=3.0.0, installed: 3.2.2]
- pbr [required: >=2.0.0,!=2.1.0, installed: 5.5.1]
- enzyme [required: >=0.4.1, installed: 0.4.1]
- guessit [required: >=3.0.0, installed: 3.1.1]
- babelfish [required: Any, installed: 0.5.5]
- python-dateutil [required: Any, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- rebulk [required: ==2.*, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- six [required: Any, installed: 1.15.0]
- pysrt [required: >=1.0.1, installed: 1.1.2]
- chardet [required: Any, installed: 3.0.4]
- pytz [required: >=2012c, installed: 2020.4]
- rarfile [required: >=2.7, installed: 4.0]
- requests [required: >=2.0, installed: <module 'pip._vendor.requests.__version__' from '/usr/lib/python3.8/site-packages/requests/__version__.py'>]
- six [required: >=1.9.0, installed: 1.15.0]
- stevedore [required: >=1.20.0, installed: 3.2.2]
- pbr [required: >=2.0.0,!=2.1.0, installed: 5.5.1]
torf==3.1.3
- flatbencode [required: ==0.2.*, installed: 0.2.1]
xdis==5.0.5
xmltodict==0.12.0
yapf==0.30.0
Thanks, that was helpful. import_module is somehow returning an incorrect object. Not sure what would be the correct fix for this as I am not able to reproduce the problem. I can push a quickfix in the meanwhile that will prevent the script from crashing.
@naiquevin I found the critical line that is affecting the version reporting by stupidly remove unrelated blocks of code and it ends up to be here:
https://github.com/naiquevin/pipdeptree/blob/be26d32c164c6b1e2e697e62a9c5a0015407d03e/pipdeptree.py#L24-L28
After importing FrozenRequirement, the version of requests reported by import_module is incorrect, this is what happens on my machine:
Python 3.9.1 (default, Dec 13 2020, 11:55:53)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import import_module
>>> import_module('requests').__version__
'2.25.0'
>>> from pip._internal.operations.freeze import FrozenRequirement
>>> import_module('requests').__version__
<module 'pip._vendor.requests.__version__' from '/usr/lib/python3.9/site-packages/requests/__version__.py'>
I don't know how to continue digging anymore. If you need me to provide anything, let me know.
Continue from the last comment, just found this
>>> import_module('requests').__version__
<module 'pip._vendor.requests.__version__' from '/usr/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests').__version__.__version__
'2.25.1'
I am not sure if it has to do with Arch Linux's packaging (I installed requests from pacman)?
This is really strange. I am not familiar with how/where pacman installs python packages. You may try the following to see which files are being loaded by import_module in both the cases
# Before importing FrozenRequirement
import_module('requests.__version__').__file__
# After importing FrozenRequirement
from pip._internal.operations.freeze import FrozenRequirement
import_module('requests.__version__').__file__
import_module('requests').__version__
Thanks for digging deeper into this.
The __file__ seems the same, here is the result. If you cannot reproduce this, it should be distro dependent since I have this problem on all my three Arch machines.
>>> from importlib import import_module
>>> import_module('requests').__version__
'2.25.1'
>>> import_module('requests.__version__').__file__
'/usr/lib/python3.9/site-packages/requests/__version__.py'
>>> import_module('requests.__version__')
<module 'requests.__version__' from '/usr/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests.__version__').__version__
'2.25.1'
>>> from pip._internal.operations.freeze import FrozenRequirement
>>> import_module('requests').__version__
<module 'pip._vendor.requests.__version__' from '/usr/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests.__version__').__version__
'2.25.1'
>>> import_module('requests.__version__').__file__
'/usr/lib/python3.9/site-packages/requests/__version__.py'
Ok. I downloaded the python-requests package for archlinux from here - https://archlinux.org/packages/extra/any/python-requests/ and extracted it. Everything seems to be correct - There's a __version__.py file and __init__.py has the following line
from .__version__ import __title__, __description__, __url__, __version__
So it should work as expected. I am unable to understand what changes after FrozenRequirement.
Anyway, I am pushing a fix to handle the case where a package contains __version__.py but isn't importing it in __init__.py. That would handle this case as well.
Actually I tried install requests in my home by (when current system package is version 2.25.1)
pip install --user requests=2.25.0
The version report is similar with system package
>>> from importlib import import_module
>>> import_module('requests').__version__
'2.25.0'
>>> import_module('requests.__version__').__file__
'/home/oliver/.local/lib/python3.9/site-packages/requests/__version__.py'
>>> import_module('requests.__version__')
<module 'requests.__version__' from '/home/oliver/.local/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests.__version__').__version__
'2.25.0'
>>> from pip._internal.operations.freeze import FrozenRequirement
>>> import_module('requests.__version__').__file__
'/home/oliver/.local/lib/python3.9/site-packages/requests/__version__.py'
>>> import_module('requests.__version__')
<module 'requests.__version__' from '/home/oliver/.local/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests').__version__
<module 'pip._vendor.requests.__version__' from '/home/oliver/.local/lib/python3.9/site-packages/requests/__version__.py'>
>>> import_module('requests.__version__').__version__
'2.25.0'
But pipdeptree works (as proved before)
$ python ./pipdeptree.py -u
qbittorrentui==0.1.3
- attrdict [required: >=2.0.0,<=2.0.1, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- blinker [required: ==1.4, installed: 1.4]
- panwid [required: ==0.2.5, installed: 0.2.5]
- blist [required: Any, installed: 1.3.6]
- orderedattrdict [required: Any, installed: 1.6.0]
- raccoon [required: ==2.1.5, installed: 2.1.5]
- blist [required: Any, installed: 1.3.6]
- six [required: Any, installed: 1.15.0]
- tabulate [required: Any, installed: 0.8.7]
- six [required: Any, installed: 1.15.0]
- urwid [required: Any, installed: 2.0.1]
- urwid-utils [required: >=0.1.2, installed: 0.1.2]
- urwid [required: Any, installed: 2.0.1]
- qbittorrent-api [required: Any, installed: 2020.12.14]
- attrdict [required: >=2.0.0, installed: 2.0.1]
- six [required: Any, installed: 1.15.0]
- requests [required: >=2.16.0, installed: 2.25.0]
- certifi [required: >=2017.4.17, installed: 2020.12.5]
- chardet [required: >=3.0.2,<4, installed: 3.0.4]
- idna [required: >=2.5,<3, installed: 2.10]
- urllib3 [required: >=1.21.1,<1.27, installed: 1.25.10]
- six [required: Any, installed: 1.15.0]
- urllib3 [required: >=1.24.2, installed: 1.25.10]
- urwid [required: ==2.0.1, installed: 2.0.1]
I am really confused. If this is too complicated to debug, it can be put off for now, I don't mind :)
I am really confused. If this is too complicated to debug, it can be put off for now, I don't mind :)
Me too :-) Yeah, let's put it off for now. I've pushed a commit in master branch that will handle the case.
Seems no update in a while, so closing. If you can still replicate please comment and we can reopen.
Just comment to say that I can not reproduce this issue now. So it's OK to close it.