Installing nose fails in `python setup.py test`, works second time through

[hartzell]

I'm working on a homework assignment for our budding Python packagers, based on this repo.

We're using Python 2.7.13 built via spack.

The script sets up working directory and a "dev virtual environment" (perhaps not strictly necessary) the clones this repo and tries to run python setup.py test.

The first time it runs it tries to install nose-cover3 twice and then complains about not being able to install nose and exits. If I run python setup.py test again it installs nose and then things work as they should.

I discovered that if I reverse the order of the test dependencies in setup.py then it succeeds, e.g.:

--- a/funniest/setup.py
+++ b/funniest/setup.py
@@ -26,7 +26,7 @@ setup(name='funniest',
           'markdown',
       ],
       test_suite='nose.collector',
-      tests_require=['nose', 'nose-cover3'],
+      tests_require=['nose-cover3', 'nose'],
       entry_points={
           'console_scripts': ['funniest-joke=funniest.command_line:main'],
       },

I'm not enough of a Python guru to understand what's going on. Perhaps nose-cover3 has messed up metadata or .... In any case it doesn't seem to be a bug here but I'm hoping you might be able to shed some light.

Below is a cut-and-paste of a sample session.

Thanks!

g.

[me@blah ~]$ # make a directory for the demo
[me@blah ~]$ DEMO_DIR=$(mktemp -t -d ${USER}-PYTHON_DEMO.XXXXX)
[me@blah ~]$
[me@blah ~]$ # set up a virtualenv for our dev work
[me@blah ~]$ # this might not be *required*, but it's safer when you're me...
[me@blah ~]$ V_DEV=${DEMO_DIR}/dev_virtenv
[me@blah ~]$ virtualenv ${V_DEV}
 test deps....  SIGH...
python setup.py test
python setup.py test
New python executable in /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/bin/python2.7
Also creating executable in /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/bin/python
Installing setuptools, pip, wheel...done.
[me@blah ~]$ source ${V_DEV}/bin/activate
(dev_virtenv) [me@blah ~]$
(dev_virtenv) [me@blah ~]$ # clone the python packaging repo (w/ funniest project)
(dev_virtenv) [me@blah ~]$ PP=${DEMO_DIR}/python-packaging
(dev_virtenv) [me@blah ~]$ git clone https://github.com/storborg/python-packaging ${PP}
Cloning into '/tmp/me-PYTHON_DEMO.tNpOB/python-packaging'...
remote: Counting objects: 167, done.
remote: Total 167 (delta 0), reused 0 (delta 0), pack-reused 167
Receiving objects: 100% (167/167), 34.13 KiB | 0 bytes/s, done.
Resolving deltas: 100% (79/79), done.
(dev_virtenv) [me@blah ~]$ cd ${PP}/funniest
(dev_virtenv) [me@blah funniest]$
(dev_virtenv) [me@blah funniest]$ # install prerequisites
(dev_virtenv) [me@blah funniest]$ python setup.py develop
running develop
running egg_info
creating funniest.egg-info
writing requirements to funniest.egg-info/requires.txt
writing funniest.egg-info/PKG-INFO
writing top-level names to funniest.egg-info/top_level.txt
writing dependency_links to funniest.egg-info/dependency_links.txt
writing entry points to funniest.egg-info/entry_points.txt
writing manifest file 'funniest.egg-info/SOURCES.txt'
reading manifest file 'funniest.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'funniest.egg-info/SOURCES.txt'
running build_ext
Creating /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/lib/python2.7/site-packages/funniest.egg-link (link to .)
Adding funniest 0.1 to easy-install.pth file
Installing funniest-joke script to /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/bin

Installed /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest
Processing dependencies for funniest==0.1
Searching for markdown
Reading https://pypi.python.org/simple/markdown/
Downloading https://pypi.python.org/packages/29/82/dfe242bcfd9eec0e7bf93a80a8f8d8515a95b980c44f5c0b45606397a423/Markdown-2.6.9.tar.gz#md5=56547d362a9abcf30955b8950b08b5e3
Best match: Markdown 2.6.9
Processing Markdown-2.6.9.tar.gz
Writing /tmp/easy_install-f7q0Sx/Markdown-2.6.9/setup.cfg
Running Markdown-2.6.9/setup.py -q bdist_egg --dist-dir /tmp/easy_install-f7q0Sx/Markdown-2.6.9/egg-dist-tmp-80aaNi
zip_safe flag not set; analyzing archive contents...
Moving Markdown-2.6.9-py2.7.egg to /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/lib/python2.7/site-packages
Adding Markdown 2.6.9 to easy-install.pth file
Installing markdown_py script to /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/bin

Installed /tmp/me-PYTHON_DEMO.tNpOB/dev_virtenv/lib/python2.7/site-packages/Markdown-2.6.9-py2.7.egg
Finished processing dependencies for funniest==0.1
(dev_virtenv) [me@blah funniest]$
(dev_virtenv) [me@blah funniest]$ # edit, test, edit, test, yay
(dev_virtenv) [me@blah funniest]$ # edit...
(dev_virtenv) [me@blah funniest]$ # install test deps fails to install nose the first time around...
(dev_virtenv) [me@blah funniest]$ # do it again and it works.  Sigh...
(dev_virtenv) [me@blah funniest]$ # or swap order of 'nose' and 'nose-cover3' in test deps....  SIGH...
(dev_virtenv) [me@blah funniest]$ python setup.py test
running test
Searching for nose-cover3
Reading https://pypi.python.org/simple/nose-cover3/
Downloading https://pypi.python.org/packages/f0/17/8c55242e86830a006bbaa0463f4a1da44f332ef7cd5a402f459c8dbaaf84/nose-cover3-0.1.0.tar.gz#md5=82f981eaa007b430679899256050fa0c
Best match: nose-cover3 0.1.0
Processing nose-cover3-0.1.0.tar.gz
Writing /tmp/easy_install-81NDHC/nose-cover3-0.1.0/setup.cfg
Running nose-cover3-0.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-81NDHC/nose-cover3-0.1.0/egg-dist-tmp-V4JbFB
warning: no previously-included files matching '*.pyc' found anywhere in distribution
creating /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg
Extracting nose_cover3-0.1.0-py2.7.egg to /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs

Installed /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg
Searching for nose
Downloading https://pypi.python.org/packages/f0/17/8c55242e86830a006bbaa0463f4a1da44f332ef7cd5a402f459c8dbaaf84/nose-cover3-0.1.0.tar.gz#md5=82f981eaa007b430679899256050fa0c
Best match: nose cover3-0.1.0
Processing nose-cover3-0.1.0.tar.gz
Writing /tmp/easy_install-OZM0ym/nose-cover3-0.1.0/setup.cfg
Running nose-cover3-0.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-OZM0ym/nose-cover3-0.1.0/egg-dist-tmp-OmgVZ0
warning: no previously-included files matching '*.pyc' found anywhere in distribution
removing '/tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg' (and everything under it)
creating /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg
Extracting nose_cover3-0.1.0-py2.7.egg to /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs

Installed /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 34, in <module>
    zip_safe=False)
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/python-2.7.13-fiaifkmalam23l42s4lyqg2kithvppfi/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/python-2.7.13-fiaifkmalam23l42s4lyqg2kithvppfi/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/python-2.7.13-fiaifkmalam23l42s4lyqg2kithvppfi/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/py-setuptools-35.0.2-xpv4en5bcykcoeorgv5ym3sklzpbow6u/lib/python2.7/site-packages/setuptools/command/test.py", line 199, in run
    installed_dists = self.install_dists(self.distribution)
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/py-setuptools-35.0.2-xpv4en5bcykcoeorgv5ym3sklzpbow6u/lib/python2.7/site-packages/setuptools/command/test.py", line 195, in install_dists
    tr_d = dist.fetch_build_eggs(dist.tests_require or [])
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/py-setuptools-35.0.2-xpv4en5bcykcoeorgv5ym3sklzpbow6u/lib/python2.7/site-packages/setuptools/dist.py", line 375, in fetch_build_eggs
    replace_conflicting=True,
  File "/sc1/apps/spack/v0.3.0/opt/spack/linux-centos7-x86_64/gcc-5.4.0/py-setuptools-35.0.2-xpv4en5bcykcoeorgv5ym3sklzpbow6u/lib/python2.7/site-packages/pkg_resources/__init__.py", line 854, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'nose' distribution was not found and is required by the application
(dev_virtenv) [me@blah funniest]$ python setup.py test
running test
Searching for nose-cover3
Best match: nose-cover3 0.1.0
Processing nose_cover3-0.1.0-py2.7.egg

Using /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose_cover3-0.1.0-py2.7.egg
Searching for nose
Reading https://pypi.python.org/simple/nose/
Downloading https://pypi.python.org/packages/58/a5/0dc93c3ec33f4e281849523a5a913fa1eea9a3068acfa754d44d88107a44/nose-1.3.7.tar.gz#md5=4d3ad0ff07b61373d2cefc89c5d0b20b
Best match: nose 1.3.7
Processing nose-1.3.7.tar.gz
Writing /tmp/easy_install-LokDOS/nose-1.3.7/setup.cfg
Running nose-1.3.7/setup.py -q bdist_egg --dist-dir /tmp/easy_install-LokDOS/nose-1.3.7/egg-dist-tmp-YF_UWz
no previously-included directories found matching 'doc/.build'
creating /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose-1.3.7-py2.7.egg
Extracting nose-1.3.7-py2.7.egg to /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs

Installed /tmp/me-PYTHON_DEMO.tNpOB/python-packaging/funniest/.eggs/nose-1.3.7-py2.7.egg
running egg_info
writing requirements to funniest.egg-info/requires.txt
writing funniest.egg-info/PKG-INFO
writing top-level names to funniest.egg-info/top_level.txt
writing dependency_links to funniest.egg-info/dependency_links.txt
writing entry points to funniest.egg-info/entry_points.txt
reading manifest file 'funniest.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'funniest.egg-info/SOURCES.txt'
running build_ext
test_basic (funniest.tests.test_command_line.TestCmd) ... ok
test_is_string (funniest.tests.test_joke.TestJoke) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.030s

OK

[storborg]

I haven't been able to replicate this issue, but I'd like to at least figure out some more clues about the problem before changing it, lest it just be some non-deterministic behavior which isn't actually fixed on certain platforms.

Could you please try something else to help debug this? Instead of doing python setup.py develop, try pip install -e .. These should behave identically, but it's possible there is some bug in setuptools which isn't present in pip.