"pip install -e ." fails in virtualenv

[certik]

Short log here:

(xx)ondrej@hawk:~/repos/clawpack(riemann_shootout)$ virtualenv-2.7 xx
New python executable in xx/bin/python
Installing setuptools............done.
Installing pip...............done.
(xx)ondrej@hawk:~/repos/clawpack(riemann_shootout)$ source xx/bin/activate
(xx)ondrej@hawk:~/repos/clawpack(riemann_shootout)$ pip install -e .
Obtaining file:///home/ondrej/repos/clawpack
  Running setup.py egg_info for package from file:///home/ondrej/repos/clawpack

Downloading/unpacking numpy>=1.6 (from clawpack==0.0.0)
  Downloading numpy-1.7.1.zip (3.1Mb): 3.1Mb downloaded
  Running setup.py egg_info for package numpy
    Running from numpy source directory.
    non-existing path in 'numpy/distutils': 'site.cfg'
    F2PY Version 2

...

    building extension "numpy.random.mtrand" sources
    C compiler: gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

    compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/home/ondrej/repos/python-2.7/xx/include/python2.7 -c'
    gcc: _configtest.c
    gcc -pthread _configtest.o -o _configtest
    _configtest
    failure.
    removing: _configtest.c _configtest.o _configtest
    building data_files sources
    build_src: building npy-pkg config files

Downloading/unpacking matplotlib>=1.0.1 (from clawpack==0.0.0)
  Downloading matplotlib-1.2.1.tar.gz (38.4Mb): 38.4Mb downloaded
  Running setup.py egg_info for package matplotlib
    basedirlist is: ['/usr/local', '/usr']
    ============================================================================
    BUILDING MATPLOTLIB
                matplotlib: 1.2.1
                    python: 2.7.3 (default, Aug  2 2012, 08:26:25)  [GCC 4.6.3]
                  platform: linux2

    REQUIRED DEPENDENCIES
                     numpy: no
                            * You must install numpy 1.4 or later to build
                            * matplotlib.
    Complete output from command python setup.py egg_info:
    basedirlist is: ['/usr/local', '/usr']

============================================================================

BUILDING MATPLOTLIB

            matplotlib: 1.2.1

                python: 2.7.3 (default, Aug  2 2012, 08:26:25)  [GCC 4.6.3]

              platform: linux2



REQUIRED DEPENDENCIES

                 numpy: no

                        * You must install numpy 1.4 or later to build

                        * matplotlib.

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /home/ondrej/repos/clawpack/xx/build/matplotlib
Storing complete log in /home/ondrej/.pip/pip.log
(xx)ondrej@hawk:~/repos/clawpack(riemann_shootout)$ 

Full log here: https://gist.github.com/certik/5475449

It doesn't compile numpy at all --- it seems it fails, yet it goes ahead and try to install matplotlib, which then of course fails, because numpy is not there. This looks like "pip" bug, but nevertheless unfortunately it prevents installing pyclaw.

[certik]

I noticed that if I do:

pip install numpy matplotlib

Then it fails as above. But if I do:

pip install numpy
pip install matplotlib

Then it works.

[ketch]

I recall seeing the same behavior previously. Is it our bug, or numpy's?

[ahmadia]

@certik, thanks for reporting this.

It turns out that this is an intentionally unfixed bug in matplotlib's setup.py script, which relies on numpy already being installed to properly interact with pip. It can be fixed with modifications similar to what I do in clawpack:

    try:
        if 'egg_info' in sys.argv:
            # only egg information for downloading requirements
            from setuptools import setup
            setuptools_dict = dict(
                install_requires = ['numpy >= 1.6',
                                    'matplotlib >= 1.0.1',
                                    ],
                extras_require = {'petclaw': ['petsc4py >= 1.2'],
                                  'euler'  : ['scipy >= 0.10.0']},
                )
            setup_dict.update(setuptools_dict)
            setup(**setup_dict)
            return

I'm not sure why the matplotlib folks have done this yet, as it simplifies the install procedure for new users.

@certik - do you think is worth trying to upstream to the matplotlib folks?

[certik]

So this piece of code should go into matplotlib's setup.py? If so, we should simply send them a pull request with the fix. As pip install numpy matplotlib failing is clearly a bug.

[goxberry]

I've also run into this bug. Definitely worth sending a fix.