boxed
Mutmut doesn't test mutants
I have a problem with mutmut: the tool creates mutants but it doesn't tests them.
Mutants using command mutmut browse:
Results:
what could be the problem??
@boxed
This is the output:
mutmut run
⠹ Generating mutants
done in 4ms
⠧ Listing all tests
⠸ Running clean tests
done
⠧ Running forced fail test
done
Running mutation testing
⠇ 380/380 🎉 0 🫥 380 ⏰ 0 🤔 0 🙁 0 🔇 0
0.00 mutations/second
They are temporary files for working with mutation testing. I bet it's the same with the cosmic ray files you have committed. At least the .sqlite files seem iffy to commit.
You have also committed .DS_Store incorrectly I se.
I'm facing something similar issue. Mutants are being created, but the tests are not being executed
Python version: 3.12.5
setup.cfg
[mutmut]
paths_to_mutate = src/
runner = pytest tests/
tests_dir = tests/
Also tested without runner at setup.cfg file
result mutmut run:
⠦ Generating mutants
done in 5ms
⠋ Listing all tests
⠴ Running clean tests
done
⠦ Running forced fail test
done
Running mutation testing
⠇ 51/51 🎉 0 🫥 51 ⏰ 0 🤔 0 🙁 0 🔇 0
0.00 mutations/second
result of mutmut results:
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_1: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_2: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_3: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_4: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_5: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_6: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_1: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_2: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_3: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_4: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_5: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_6: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_7: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_8: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_9: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_10: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_11: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_12: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_13: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_1: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_2: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_3: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_4: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_5: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_6: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_7: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_8: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_9: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_10: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_11: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_12: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_13: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_14: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_15: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_16: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_17: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_18: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_19: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_20: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_21: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_22: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_23: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_24: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_25: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_26: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_27: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_28: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_29: no tests
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_30: no tests
domain.model.exception.customException.xǁNomeInvalidoErrorǁ__init____mutmut_1: no tests
domain.model.exception.customException.xǁEmailInvalidoErrorǁ__init____mutmut_1: no tests
coverage report:
Name Stmts Miss Cover
-------------------------------------------------------------------
src/__init__.py 0 0 100%
src/domain/__init__.py 0 0 100%
src/domain/model/__init__.py 0 0 100%
src/domain/model/exception/__init__.py 0 0 100%
src/domain/model/exception/customException.py 6 0 100%
src/domain/model/usuario/__init__.py 0 0 100%
src/domain/model/usuario/usuario.py 68 0 100%
tests/__init__.py 0 0 100%
tests/domain/__init__.py 0 0 100%
tests/domain/model/__init__.py 0 0 100%
tests/domain/model/usuario/__init__.py 0 0 100%
tests/domain/model/usuario/usuario_test.py 101 0 100%
-------------------------------------------------------------------
TOTAL 175 0 100%
When I run mutmut with debug=True on setup.cfg I was able to see the problem:
result = _mutmut_trampoline(object.__getattribute__(self, "xǁUsuarioǁ__init____mutmut_orig"), object.__getattribute__(self, "xǁUsuarioǁ__init____mutmut_mutants"), *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
orig = <bound method Usuario.xǁUsuarioǁ__init____mutmut_orig of <src.domain.model.usuario.usuario.Usuario object at 0x110d86c00>>
mutants = {'xǁUsuarioǁ__init____mutmut_1': <function Usuario.xǁUsuarioǁ__init____mutmut_1 at 0x110dfa2a0>, 'xǁUsuarioǁ__init____...3 at 0x110dfa700>, 'xǁUsuarioǁ__init____mutmut_4': <function Usuario.xǁUsuarioǁ__init____mutmut_4 at 0x110dfa840>, ...}
args = ('Micci', '[email protected]'), kwargs = {}, os = <module 'os' (frozen)>, mutant_under_test = 'fail', MutmutProgrammaticFailException = <class 'mutmut.__main__.MutmutProgrammaticFailException'>
def _mutmut_trampoline(orig, mutants, *args, **kwargs):
import os
mutant_under_test = os.environ['MUTANT_UNDER_TEST']
if mutant_under_test == 'fail':
from mutmut.__main__ import MutmutProgrammaticFailException
> raise MutmutProgrammaticFailException('Failed programmatically')
E mutmut.__main__.MutmutProgrammaticFailException: Failed programmatically
But I'm not understanding why mutmut is crashing
Anyway solved doing a downgrade on mutmut 3 to mutmut 2.5.1
@vinceslao maybe this could work for you too
mutmut run
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Results are stored in .mutmut-cache.
Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests need to be expanded.
🔇 Skipped. Skipped.
1. Running tests without mutations
⠋ Running...Done
2. Checking mutants
⠧ 75/75 🎉 43 ⏰ 0 🤔 5 🙁 27 🔇 0
@MrMicc MutmutProgrammaticFailException is on purpose. If it doesn't manage to do that, it will fail for real, so that's not it. Is that project available so I can test?
Hi @boxed ! Happy New Year! 🥳
Link to repo: https://github.com/MrMicc/Study/tree/main/python-playground
Thanks in advance
@MrMicc The code you have there is not correct. https://github.com/MrMicc/Study/blob/main/python-playground/tests/domain/model/usuario/usuario_test.py has imports like from src..... They should not begin with src.. src should not be a module path.
I'm continuing the discussion here (started here for information https://github.com/boxed/mutmut/issues/349#issuecomment-2568398062 ) because my problem now seems more like the one presented here.
I've updated my repository https://github.com/Julien-Delavisse/mutmut-v3-tests with a test_single.sh script to test from a mutmut github commit (using docker for better reproducibility).
I'm getting a slightly different result with this script than with my system :
⠋ Generating mutantssrc/helloworld.py
also copying tests
also copying test
also copying tests.py
also copying setup.cfg
also copying pyproject.toml
done in 13ms
⠙ Running stats python -m pytest -vv -x -q --import-mode=append --rootdir=.
============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /usr/local/bin/python3.12
cachedir: .pytest_cache
rootdir: /app/mutants
configfile: pyproject.toml
testpaths: tests
collecting ... collected 1 item
tests/test_helloworld.py::test_hello_world PASSED [100%]
============================== 1 passed in 0.00s ===============================
exit code 0
done
⠹ Running clean testspython -m pytest -vv -x -q --import-mode=append --rootdir=.
============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /usr/local/bin/python3.12
cachedir: .pytest_cache
rootdir: /app/mutants
configfile: pyproject.toml
testpaths: tests
collecting ... collected 1 item
tests/test_helloworld.py::test_hello_world PASSED [100%]
============================== 1 passed in 0.00s ===============================
exit code 0
done
⠸ Running forced fail testpython -m pytest -vv -x -q --import-mode=append --rootdir=.
============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /usr/local/bin/python3.12
cachedir: .pytest_cache
rootdir: /app/mutants
configfile: pyproject.toml
testpaths: tests
collecting ... collected 1 item
tests/test_helloworld.py::test_hello_world FAILED [100%]
=================================== FAILURES ===================================
_______________________________ test_hello_world _______________________________
def test_hello_world():
> assert helloworld.hello_world() == 1
tests/test_helloworld.py:4:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/helloworld.py:56: in hello_world
result = _mutmut_trampoline(x_hello_world__mutmut_orig, x_hello_world__mutmut_mutants, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
orig = <function x_hello_world__mutmut_orig at 0x7314fdcfcf40>
mutants = {'x_hello_world__mutmut_1': <function x_hello_world__mutmut_1 at 0x7314fdcfcfe0>}
args = (), kwargs = {}, os = <module 'os' (frozen)>, mutant_under_test = 'fail'
MutmutProgrammaticFailException = <class 'mutmut.__main__.MutmutProgrammaticFailException'>
def _mutmut_trampoline(orig, mutants, *args, **kwargs):
import os
mutant_under_test = os.environ['MUTANT_UNDER_TEST']
if mutant_under_test == 'fail':
from mutmut.__main__ import MutmutProgrammaticFailException
> raise MutmutProgrammaticFailException('Failed programmatically')
E mutmut.__main__.MutmutProgrammaticFailException: Failed programmatically
src/helloworld.py:9: MutmutProgrammaticFailException
=========================== short test summary info ============================
FAILED tests/test_helloworld.py::test_hello_world - mutmut.__main__.MutmutPro...
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
============================== 1 failed in 0.01s ===============================
exit code 1
done
Running mutation testing
⠼ 0/1 🎉 0 🫥 0 ⏰ 0 🤔 0 🙁 0 🔇 0python -m pytest -vv -x -q --import-mode=append tests/test_helloworld.py::test_hello_world --rootdir=.
============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /usr/local/bin/python3.12
cachedir: .pytest_cache
rootdir: /app/mutants
configfile: pyproject.toml
collecting ... collected 1 item
tests/test_helloworld.py::test_hello_world FAILED [100%] worker exit code 1
⠴ 1/1 🎉 1 🫥 0 ⏰ 0 🤔 0 🙁 0 🔇 0
44.64 mutations/second
@MrMicc The code you have there is not correct. https://github.com/MrMicc/Study/blob/main/python-playground/tests/domain/model/usuario/usuario_test.py has imports like
from src..... They should not begin withsrc..srcshould not be a module path.
Hey @boxed thanks for your support!
I didnt know about src, sorry about it! I'm a newbie at python 😅
(Ah.. if want I can add this for you at the README)
I configure pytest.ini
[pytest]
pythonpath = src
But now I'm receiving a problema similar from the issue #341
when I do mutmut run
======================================================================================= 50 passed in 0.02s =======================================================================================
exit code 0
FAILED
with mutmut results returning:
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_1: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_2: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_3: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_4: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_5: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__init____mutmut_6: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_1: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_2: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_3: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_4: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_5: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_6: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_7: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_8: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_9: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_10: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_11: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_12: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_nome_valido__mutmut_13: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_1: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_2: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_3: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_4: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_5: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_6: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_7: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_8: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_9: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_10: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_11: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_12: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_13: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_14: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_15: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_16: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_17: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_18: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_19: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_20: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_21: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_22: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_23: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_24: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_25: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_26: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_27: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_28: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_29: not checked
domain.model.usuario.usuario.xǁUsuarioǁ__checa_se_email_valido__mutmut_30: not checked
domain.model.exception.customException.xǁNomeInvalidoErrorǁ__init____mutmut_1: not checked
domain.model.exception.customException.xǁEmailInvalidoErrorǁ__init____mutmut_1: not checked
I'm getting a slightly different result with this script than with my system :
Yea. 100% mutants killed. 100% success :P @Julien-Delavisse
@MrMicc We should not add this information to the readme about how to use python. But I've added a check for it in mutmut so it will fail with an error message. It seems like university courses use mutmut, and there the level of knowledge about python is not super high. This is not really the use case I envisioned when I created mutmut 😅
@MrMicc We should not add this information to the readme about how to use python. But I've added a check for it in mutmut so it will fail with an error message. It seems like university courses use mutmut, and there the level of knowledge about python is not super high. This is not really the use case I envisioned when I created mutmut 😅
hahaha I got your point! That is what happened when you make something really useful 😄
...There's no better way to learn something than through unit tests, and with mutation testing, it's even better
Congrats about it! It is a really nice and powerful toll
Having the same issue, mutmut is skipping the mutation tests
https://github.com/GH-maggio/hello_mutmut
uv run mutmut run
❯ uv run mutmut run
Built hello-mutmut @ file:///home/cmaggio/repos/hello_mutmut
Uninstalled 1 package in 0.59ms
Installed 1 package in 1ms
⠋ Generating mutantscomponents/hello_mutmut/apple/core.py
components/hello_mutmut/apple/__init__.py
also copying tests
also copying test
also copying setup.cfg
also copying pyproject.toml
done in 2ms
⠙ Listing all tests python -m pytest -vv -x -q --collect-only --rootdir=.
======================================================= test session starts ========================================================
platform linux -- Python 3.13.2, pytest-8.3.4, pluggy-1.5.0 -- /home/cmaggio/repos/hello_mutmut/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/cmaggio/repos/hello_mutmut/mutants
configfile: pyproject.toml
collected 3 items
<Dir mutants>
<Dir test>
<Dir components>
<Dir hello_mutmut>
<Package apple>
<Module test_core.py>
<Function test_add_apples>
<Function test_con_apples>
<Function test_count_apples>
==================================================== 3 tests collected in 0.01s ====================================================
exit code 0
⠹ Running clean testspython -m pytest -vv -x -q --import-mode=append --rootdir=.
======================================================= test session starts ========================================================
platform linux -- Python 3.13.2, pytest-8.3.4, pluggy-1.5.0 -- /home/cmaggio/repos/hello_mutmut/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/cmaggio/repos/hello_mutmut/mutants
configfile: pyproject.toml
collected 3 items
test/components/hello_mutmut/apple/test_core.py::test_add_apples PASSED [ 33%]
test/components/hello_mutmut/apple/test_core.py::test_con_apples PASSED [ 66%]
test/components/hello_mutmut/apple/test_core.py::test_count_apples PASSED [100%]
======================================================== 3 passed in 0.01s =========================================================
exit code 0
done
⠸ Running forced fail testpython -m pytest -vv -x -q --import-mode=append --rootdir=.
======================================================= test session starts ========================================================
platform linux -- Python 3.13.2, pytest-8.3.4, pluggy-1.5.0 -- /home/cmaggio/repos/hello_mutmut/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/cmaggio/repos/hello_mutmut/mutants
configfile: pyproject.toml
collected 3 items
test/components/hello_mutmut/apple/test_core.py::test_add_apples PASSED [ 33%]
test/components/hello_mutmut/apple/test_core.py::test_con_apples PASSED [ 66%]
test/components/hello_mutmut/apple/test_core.py::test_count_apples PASSED [100%]
======================================================== 3 passed in 0.01s =========================================================
exit code 0
FAILED
mutmut==2.51 runs successfully
❯ uv run mutmut run --paths-to-mutate bases/hello_mutmut,components/hello_mutmut
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Results are stored in .mutmut-cache.
Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests need to be expanded.
🔇 Skipped. Skipped.
1. Using cached time for baseline tests, to run baseline again delete the cache file
2. Checking mutants
⠙ 1/1 🎉 1 ⏰ 0 🤔 0 🙁 0 🔇 0
Having the same issue,
mutmutis skipping the mutation testshttps://github.com/GH-maggio/hello_mutmut
I am having exactly the same issue when I migrated to mutmut3.
In mutmut2, I ran the command mutmut run --paths-to-mutate common/service/ --tests-dir test/ and everything is working properly.
When I migrated to mutmut3, I have setup.cfg as follow
[mutmut]
paths_to_mutate=common/service/
tests_dir=test/
When I run mutmut run it successfully detect the tests and mutates my source file, I see the mutant folder being created. But it seems that the generated tests under mutant folder are not attached to mutated source code.
I think there might have a bug with multi level directory detection in mutmut3 (or bug from python not able to associate mutant tests with mutant source). If you flatten your project structure, it should work.
I see mutants directory being generated which contains all the mutated source code under common/service. Since I don't have complex hierarchy in my test folder, all the tests under test are also generated.
I have the feeling that generated tests doesn't have visibility to generated mutants source. This is simply one of my hypothesis. Seems quite a few people, including @vinceslao and @GH-maggio, have similar problem.
@ytqxyd
all the tests under test are also generated.
You mean mutants were generated for the tests? In that case it is the same issue as I mentioned. Tests should not be mutated.
You mean mutants were generated for the tests? In that case it is the same issue as I mentioned. Tests should not be mutated.
However, there's no mutation on the tests. It is simply a copy paste from my original tests into mutants directory. I do see mutation on mutants/common/service. Is this behaviour unexpected?
@boxed In this scenario, original tests and mutants are generated in mutants directory.
However, when I run mutmut run, it doesn't run tests which test mutants. Which is really odd.
In mutmut2, with exactly the same . I don't think the root cause is related to #368.
This bug has been opened since end of 2024, do we have any lead on what might cause it? I can take a look at the source code if you can point out a potential root cause.
Thank you and really appreciate you maintaining this package
@ytqxyd if it fails on the forced fail part, then it's probably a python path issue. I don't really have much time to work on mutmut. I got a few weeks between jobs so that's why there was suddenly a huge spike in mutmut development and I could finally make v3 a reality, but now I need very small and clean reproduction scenarios to have any chance of looking at bugs. I would love if you could take a look!