scrapy-webdriver
scrapy-webdriver copied to clipboard
distribute installation breaks pip, fabric, etc.
Why are you including distribute 0.6.27 with this package? It makes no sense and is breaking pip and fabric after installing this package. For example:
$ fab
Traceback (most recent call last):
File "/usr/local/bin/fab", line 5, in <module>
from pkg_resources import load_entry_point
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2735, in <module>
return mkstemp(*args,**kw)
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 690, in require
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 588, in resolve
Unless `replace_conflicting=True`, raises a VersionConflict exception if
pkg_resources.DistributionNotFound: paramiko>=1.10
$ pip install paramiko
Downloading/unpacking paramiko
Downloading paramiko-1.15.1-py2.py3-none-any.whl (165kB): 165kB downloaded
Cleaning up...
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1259, in prepare_files
)[0]
IndexError: list index out of range
Storing debug log for failure in /root/.pip/pip.log
These all seem like bugs caused by using the old version of distribute found in this package. I don't see a reason to ever include distribute with your package which will overwrite the user's own, but perhaps I'm missing something.
I actually used a packaging bootstrapping template this time around. Probably not the best of choices. I wonder in which circumstances including distribute breaks pip though, as the initial project that prompted building scrapy-webdriver in the first place was deployed using pip. Regardless, thanks for the pull request, I'll make sure to take a look ASAP.
Yeah, I didn't have this issue when I was using scrapy-webdriver within a virtualenv, but it appeared when I tried using it in Docker (without virtualenv). Presumably because it overwrites the system's distribute, though I can't say for sure. Nowadays distribute is merely a wrapper for setuptools anyway, so I think there's no harm in removing it from your package.
I didn't create a PR as I wasn't sure if you'd be interested, but it fixed the issue for me, so feel free to merge it in if you want. :)