buildout pip fails to install zc.recipe.egg wheel

I'm getting this error on multiple environments (Mac, Ubuntu 14.04, Redhat 6.1) when trying to install / upgrade to the latest version of zc.recipe.egg (am I posting this in the right place?) 'pip', 'wheel' and 'virtualenv' packages are all up to date.

> pip install zc.recipe.egg
Collecting zc.recipe.egg
  Downloading zc.recipe.egg-2.0.3-py2.py3-none-any.whl
Installing collected packages: zc.recipe.egg
zc.recipe.egg is in an unsupported or invalid wheel

Oct 06 '15 11:10 wiggin15

I can reproduce.

I've no idea what it means.

Oct 06 '15 11:10 mgedmin

I can download and unzip the file just fine, no errors. If I look at zc.recipe.egg-2.0.3.dist-info/WHEEL it looks like this:

Wheel-Version: 1.0
Generator: bdist_wheel (0.24.0)
Root-Is-Purelib: true
Tag: py2-none-any
Tag: py3-none-any

I don't know much about wheels, but purely from looking at this, it looks fine. Pure python, python 2 and 3. Generated with setuptools ("bdist_wheel") and uploaded with twine. Both setuptools and twine were called by zest.releaser and that's being used a lot for this task. So...

Also: I released zc.buildout in (to me at least) exactly the same way and a pip install of the buildout wheel works fine.

So... does anyone know what's wrong?

Oct 06 '15 11:10 reinout

Running pip with -vvvvv gives me:

Installing collected packages: zc.recipe.egg
Cleaning up...
zc.recipe.egg is in an unsupported or invalid wheel
Exception information:
Traceback (most recent call last):
  File "/private/tmp/reutel/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/private/tmp/reutel/lib/python2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/private/tmp/reutel/lib/python2.7/site-packages/pip/req/req_set.py", line 646, in install
    **kwargs
  File "/private/tmp/reutel/lib/python2.7/site-packages/pip/req/req_install.py", line 801, in install
    pip.wheel.check_compatibility(version, self.name)
  File "/private/tmp/reutel/lib/python2.7/site-packages/pip/wheel.py", line 576, in check_compatibility
    "%s is in an unsupported or invalid wheel" % name
UnsupportedWheel: zc.recipe.egg is in an unsupported or invalid wheel

Oct 06 '15 11:10 reinout

I think this might be a bug in pip -- it gets confused by the name ending with ".egg".

mg@platonas: /tmp $ pip install ./zc.recipe.egg-2.0.3-py2.py3-none-any.whl 
Processing ./zc.recipe.egg-2.0.3-py2.py3-none-any.whl
Installing collected packages: zc.recipe.egg
zc.recipe.egg is in an unsupported or invalid wheel

mg@platonas: /tmp $ cp zc.recipe.egg-2.0.3-py2.py3-none-any.whl zc.recipe.eggs-2.0.3-py2.py3-none-any.whl

[exited with 1]
mg@platonas: /tmp $ pip install ./zc.recipe.eggs-2.0.3-py2.py3-none-any.whl 
Processing ./zc.recipe.eggs-2.0.3-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/mg/.venv/lib/python2.7/site-packages (from zc.recipe.eggs==2.0.3)
Collecting zc.buildout>=1.2.0 (from zc.recipe.eggs==2.0.3)
  Using cached zc.buildout-2.4.4-py2.py3-none-any.whl
Installing collected packages: zc.buildout, zc.recipe.eggs
Exception:
Traceback (most recent call last):
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/req/req_set.py", line 646, in install
    **kwargs
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 803, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 998, in move_wheel_files
    isolated=self.isolated,
  File "/home/mg/.venv/local/lib/python2.7/site-packages/pip/wheel.py", line 341, in move_wheel_files
    assert info_dir, "%s .dist-info directory not found" % req
AssertionError: zc.recipe.eggs==2.0.3 .dist-info directory not found

[exited with 2]

Oct 06 '15 12:10 mgedmin

Well done that you've found it! Thanks. Are you reporting this to pip, too?

Oct 06 '15 12:10 reinout

On IRC, yes.

(It might be a setuptools bug, rather than pip's, but anyway, it doesn't look like anything we're doing wrong here, other than picking an unfortunate package name ;-)

Oct 06 '15 12:10 mgedmin

Please note that a similar issue has been reported to pip here: https://github.com/pypa/pip/issues/3028 (I can't explain how he tried to install a wheel version of this package before 2.0.3 was released. Maybe he built one for Windows only)

Oct 06 '15 12:10 wiggin15

I suggest removing the wheel from PyPI until this gets resolved.

I also suggest removing whatever config bits tell zest.releaser to upload wheels, since if this needs to be fixed in pip/setuptools, it'll take a while until all users upgrade.

Oct 06 '15 12:10 mgedmin

I have removed the wheel from PyPI.

Oct 06 '15 12:10 mgedmin

Should we leave the zc.buildout wheel on pypi, then? zc.buildout is useless without zc.recipe.egg.

Oct 06 '15 12:10 reinout

@mgedmin: I switched off automatic wheel building here: https://github.com/buildout/buildout/commit/cc9cc8a27de2df01c1e3dbeff26c669de52de86f

Oct 06 '15 12:10 reinout

Unfortunately removing the wheel doesn't help any: pip 7.x installs source packages by running setup.py bdist_wheel and then installing the resulting whl file. This fails.

Oct 06 '15 12:10 mgedmin

There's no reason to remove zc.buildout wheels from PyPI.

Oct 06 '15 12:10 mgedmin

IRC:

<dstufft> mgedmin: you'd need to modify your setup.py so that bdist_wheel raises an error
<dstufft> that'll force pip to fall back to setup.py install

Oct 06 '15 12:10 mgedmin

FWIW pip install zc.recipe.egg==2.0.2 also fails with the same error, so it's just a coincidence that this bug showed up right after the universal wheel support got merged.

Oct 06 '15 12:10 mgedmin

pip install zc.recipe.egg==2.0.2 works correctly on my environment. Try to run pip with --no-cache-dir

Oct 06 '15 12:10 wiggin15

@wiggin15: what is your pip's version?

Oct 06 '15 12:10 reinout

@wiggin15 what pip version? I wiped my ~/.cache/pip/http before experimenting, and I checked to see that ~/.cache/pip/wheels/*/*/*/*/ did not have anything named zc.recipe.egg*. You can see in my traceback that pip was downloading the 2.0.2 tarball and running setup.py bdist_wheel on it.

Oct 06 '15 12:10 mgedmin

Ok, as far as I understand:

this problem affects every user of pip 7.x or newer (EDIT: if they have wheel installed and don't disable the wheel cache)
this problem (used to) affect users of older pip versions when PyPI had the .whl file

So there was some use of my removing the .whl file from PyPI: at least I unbroke the situation for users of outdated software :)

Oct 06 '15 12:10 mgedmin

Latest pip, 7.1.2. Here's the full output -- now it works on my Mac only with --no-cache-dir, but on a clean VM it works without this flag too. https://gist.github.com/wiggin15/cefeef67e127ecbbfb95

Oct 06 '15 12:10 wiggin15

I would like to see the pip install -vvv output of it working in a clean VM without --no-cache-dir and with pip 7.x.

Oct 06 '15 12:10 mgedmin

I take it back, sorry! Looks like the VM wasn't entirely clean. It works with -U or with --no-cache-dir, but not on a really clean environment. I don't know how to explain. FWIW, here's the output with -vvv and -U: https://gist.github.com/wiggin15/2718ed5cce8df1b7804f

Oct 06 '15 13:10 wiggin15

Disregard. 'wheel' package wasn't installed on the VM. doh

Oct 06 '15 13:10 wiggin15

Thank you!

Oct 07 '15 05:10 mgedmin

The stable build of the Zope RTD docs just now failed due to this bug.