On Travis, mpi is not installed correctly

[ketch]

In this build the parallel tests on Travis are failing. In Python 3, we get command not found when executing mpirun. In Python 2, the tests run for a ridiculously long time and then we get

ERROR: pyclaw.examples.psystem_2d.psystem_2d({'tfinal': 1.0, 'outdir': './_for_temp_pyclaw_test', 'cells_per_layer': 6, 'solver_type': 'classic'})
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/clawpack/pyclaw/clawpack/stack/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/clawpack/pyclaw/clawpack/stack/lib/python2.7/site-packages/clawpack/pyclaw/util.py", line 175, in <lambda>
    test = lambda: test_app(application, verifier, test_kwargs)
  File "/home/travis/build/clawpack/pyclaw/clawpack/stack/lib/python2.7/site-packages/clawpack/pyclaw/util.py", line 241, in test_app
    check_values = verifier(claw)
  File "/home/travis/build/clawpack/pyclaw/src/pyclaw/examples/psystem_2d/test_2d_psystem.py", line 36, in verify
    reltol=1e-4)
  File "/home/travis/build/clawpack/pyclaw/clawpack/stack/lib/python2.7/site-packages/clawpack/pyclaw/util.py", line 276, in check_diff
    err_norm = d*np.linalg.norm(expected - test)
ValueError: operands could not be broadcast together with shapes (24,3) (0,) 
-------------------- >> begin captured stdout << ---------------------
{'use_petsc': True, 'tfinal': 1.0, 'kernel_language': 'Fortran', 'solver_type': 'classic', 'cells_per_layer': 6, 'outdir': './_for_temp_pyclaw_test'}

This appears to be an installation issue triggered by adding pip install h5py to the installation script.

[mandli]

Is hdf5 installed on travis by default? If not we need to do a apt-get call.

[ketch]

I believe it is installed for us by hashstack (not default). But for some reason Python 3 doesn't see it unless I add the pip install line. We could try apt-get instead.

[mandli]

Hmm, had forgotten about that. If we can keep it using hashstack that would be ideal but if that breaks in the future are we going to be able to fix it ourselves?

[ketch]

No, I have no idea how to fix it. I haven't started looking since I am still hoping for help from @ahmadia . It might be wise to switch to something we understand better, but we could never get all the dependencies to work together with just apt-get/pip (at least, back in 2013/14).

[ahmadia]

You can't mix and match Python packages between apt-get/hashstack, though you can mix-and-match dependencies (such as HDF5) if you're careful.

Update for clarity: You can mix-and-match between hashstack and pip (once the pip points to a hashstack Python), but you generally can't use an apt-get installed package from the hashstack Python.

[ahmadia]

This is the file that needs h5py: https://github.com/clawpack/pyclaw/blob/master/.stack.yaml - it appears to be in here, so we'd need to debug why the h5py package in hashstack doesn't show up in Python 3.

[ahmadia]

This looks fine to me: https://github.com/hashdist/hashstack/blob/master/pkgs/h5py.yaml

[ahmadia]

I don't know how to fix this without putting together a local Travis image then debugging what's gone wrong in the build: https://docs.travis-ci.com/user/common-build-problems/#Running-a-Container-Based-Docker-Image-Locally

In principle, we had h5 support working, though I don't know if it was ever tested with Python 3. I don't know why the mpirun command is failing either.

[ahmadia]

And it looks like I'm wrong, you should be able to install h5py from outside using pip. In that case it might make sense to remove it from the hashstack file (.stack.yaml) in case you're getting conflicts when trying to install.

[ahmadia]

@ketch - for some reason I can't PR to your repository, but try replacing h5py with hdf5 in the .stack.yaml file.