[BUG] unit tests fails if the active profile is not `default`
Describe the bug
when the local profile is set to test as the active one, unittests of test_utils.py fail when running all tests in one pytest session. If the tests are run individually, the tests will pass. This can be misleading for developers and time-consuming. Besides, it seems that something is changing globally during tests, which is incorrect.
To Reproduce Steps to reproduce the behavior:
- activate the
testprofile - run
pytest
Additional context
The problem is still not 100% clear to me, however, a couple of clues:
- Deleting
test_profile.pyresolves the issue. - The problem starts to happen after merging this PR: #342
- This line
from medperf.commands.profile import appintest_profile.pyseems to be the issue. When commenting all the code in this file and keeping this import statement, the problem persists.
Possible explanation: After merging the mentioned PR, the decorator configurable was edited to modify config. When importing app in test_profile.py, the decorator seems to be executed. However, it is still not clear why test_utils.py is failing.
Solution tried but it didn't work: adding this fixture in conftest.py
@pytest.fixture(autouse=True)
def disable_set_custom_config(mocker):
mocker.patch("medperf.utils.set_custom_config")
Note Good to keep this in mind if we encountered any weird unit-tests failures.
could have been fixed by https://github.com/mlcommons/medperf/pull/516. confirmation required
At least for me it works well since that PR, so confirm it is fixed:)