Fix KLU not installed error in validation repository

[Vaibhav-Chopra-GT]

Description

I tried everything, but installing pybamm in GitHub Actions using pip always throws the KLU not installed error. I tried creating a separate tox file (as that works for pybamm right now) for the validation repository, but it still did not work. A simple fix for this was to keep the scripts inside the pybamm repository, clone pybamm in validation repository's workflow, and run the scripts in pybamm's tox environment.

I have made the required changes in this PR, and the workflows are passing here - https://github.com/Vaibhav-Chopra-GT/test-pybamm/actions/runs/3032880848

Fixes # (issue)

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

• [ ] New feature (non-breaking change which adds functionality)
• [ ] Optimization (back-end change that speeds up the code)
• [ ] Bug fix (non-breaking change which fixes an issue)

Key checklist:

• [ ] No style issues: $ flake8
• [ ] All tests pass: $ python run-tests.py --unit
• [ ] The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.

Further checks:

• [ ] Code is commented, particularly in hard-to-understand areas
• [ ] Tests added that prove fix is effective or that feature works

[codecov[bot]]

Codecov Report

Merging #2297 (e3fad88) into develop (b25ce8c) will increase coverage by 0.00%. The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #2297   +/-   ##
========================================
  Coverage    99.39%   99.40%           
========================================
  Files          364      364           
  Lines        19996    20034   +38     
========================================
+ Hits         19876    19914   +38     
  Misses         120      120           

Impacted Files	Coverage Δ
pybamm/simulation.py	98.86% <0.00%> (ø)
pybamm/solvers/base_solver.py	100.00% <0.00%> (ø)
pybamm/expression_tree/functions.py	100.00% <0.00%> (ø)
pybamm/models/submodels/interface/sei/base_sei.py	100.00% <0.00%> (ø)
...ybamm/models/submodels/interface/base_interface.py	100.00% <0.00%> (ø)
...s/full_battery_models/lithium_ion/electrode_soh.py	100.00% <0.00%> (ø)
...m/models/full_battery_models/base_battery_model.py	99.78% <0.00%> (+<0.01%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[Saransh-cpp]

This is weird. I created a minimal example - https://github.com/Saransh-cpp/pybamm-idaklusolver-test/blob/main/.github/workflows/ci.yml - and it indeed does not work - https://github.com/Saransh-cpp/pybamm-idaklusolver-test/actions/runs/3115076483/jobs/5051607653

Maybe @martinjrobins can help?

[valentinsulzer]

@Saransh-cpp can you try your workflow with the branch idaklu-install-debug? To see which part of importing IDA KLU is failing

[Saransh-cpp]

The workflow outputs -

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/hostedtoolcache/Python/3.9.[14](https://github.com/Saransh-cpp/pybamm-idaklusolver-test/actions/runs/3115467411/jobs/5052389760#step:8:15)/x64/lib/python3.9/site-packages/pybamm/solvers/idaklu_solver.py", line 59, in __init__
    raise ImportError(f"KLU is not installed: {idaklu_spec}")
ImportError: KLU is not installed: Could not import idaklu module

[valentinsulzer]

Can you run again? Just changed it so it saves the ImportError message

[Saransh-cpp]

A missing file?

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/hostedtoolcache/Python/3.9.[14](https://github.com/Saransh-cpp/pybamm-idaklusolver-test/actions/runs/3115508691/jobs/5052473330#step:8:15)/x64/lib/python3.9/site-packages/pybamm/solvers/idaklu_solver.py", line 59, in __init__
    raise ImportError(f"KLU is not installed: {idaklu_spec}")
ImportError: KLU is not installed: Could not import idaklu module: libsundials_sunmatrixsparse.so.3: cannot open shared object file: No such file or directory

[valentinsulzer]

Could be an issue with LD_LIBRARY_PATH (see here)

This is set (added to the environment activate file) by tox.ini in pybamm: https://github.com/pybamm-team/PyBaMM/blob/f5d16bc424af09bc0b4c0879801cbe7d88bf4807/tox.ini#L27

[valentinsulzer]

Having said that: since we are moving to a monorepo (#2315 ) we should just do all this in a new folder validation in this repo

[Saransh-cpp]

Adding the missing env variables worked!

The passing workflow - https://github.com/Saransh-cpp/pybamm-idaklusolver-test/actions/runs/3137300340 YAML file - https://github.com/Saransh-cpp/pybamm-idaklusolver-test/blob/main/.github/workflows/ci.yml

@Vaibhav-Chopra-GT, you can set the required variables in validation repository's workflows and then move all the related benchmarks there -

    - name: Clone PyBaMM
      run: |
        python -m pip install wheel
        git clone -b develop --single-branch https://github.com/pybamm-team/PyBaMM
    - name: Install casadi cmake numpy
      run: python -m pip install cmake casadi numpy
    - name: Install Sundials, KLU, and clone pybind11
      run: |
        pip install wget
        cd PyBaMM
        python ./scripts/install_KLU_Sundials.py
        git clone https://github.com/pybind/pybind11.git ./pybind11
    - name: Set env variables and install PyBaMM
      run: |
        echo "SUNDIALS_INST=$HOME/.local" >> $GITHUB_ENV
        echo "LD_LIBRARY_PATH=$HOME/.local/lib" >> $GITHUB_ENV
        cd PyBaMM
        python -m pip install . 
    - name: Test
      run: python -c "import pybamm; pybamm.IDAKLUSolver()"