MUSE_OS

MUSE_OS copied to clipboard

Describe the bug

The developer documentation instructs to use python -m pytest to run the tests. However, I've found that the tests fail when using this command (more details below). I note that this command isn't actually run as part of the CI, which first runs the tests excluding the notebook tests and the regression tests, before running the regression tests separately. Splitting the tests up in this way also works fine locally. Nevertheless, the tests should ideally pass when running the full suite. Otherwise, the developer instructions should change to clarify which tests should be run locally and which shouldn't.

Details:

• Two regression tests fail when running the full test suite with the command above. Here's the error:

FAILED tests/test_fullsim_regression.py::test_tutorial_regression[tutorial_path10] - KeyError: "['timeslice', 'asset', 'year', 'commodity'] not in index"
FAILED tests/test_fullsim_regression.py::test_tutorial_regression[tutorial_path11] - KeyError: "['timeslice', 'asset', 'year', 'commodity'] not in index"

• However, these test pass when running pytest tests/test_fullsim_regression.py
• pytest -m "not regression and not notebook" passes
• I've also noticed that various files for simulation results in docs/advanced-guide are changed during the tests. Why are these files changing? Should these changes be committed? The notebooks extending-muse.ipynb and further-extending-muse.ipynb appears to be the culprits.
• Why aren't notebook tests being run as part of CI?
• I think the models that are failing are 0-new-decision-metric and 3-add-region

To Reproduce

I ran the following:

python -m venv .venv
.venv/scripts/Activate.ps1
pip install -e .[dev,doc]
python -m pytest

Expected behavior

Ideally the tests should pass when running the full suite. Otherwise, the developer instructions should change to clarify which tests should be run locally

Context

Please, complete the following to better understand the system you are using to run MUSE.

• Operative system (eg. Windows 10): I've tried on Windows 10
• MUSE version (eg. 1.0.1): dev branch
• Installation method (eg. pipx, pip, development mode): dev mode
• Python version (you can get this running python --version): 3.9.18