mutmut icon indicating copy to clipboard operation
mutmut copied to clipboard

Mutmut tests on windows is broken

Open boxed opened this issue 5 years ago • 8 comments

Hey @nklapste maybe you can take a look at this? I don't have a windows machine so I have to push to travis to try to figure this out and it's no fun at all.

On a brighter note, the new mutmut is suuuuuuper cool if you use it with hammett! You should try it out!

boxed avatar Mar 26 '20 08:03 boxed

@boxed Mutmut seems to work for me on Windows when not setting runner in setup.cfg. However, when setting runner to runner=python -m hammett -x -q src/tests then none of the mutants seem to get killed even if they should be. Am I setting the runner wrong?

Using Python 3.7.4

DanJBower avatar Sep 09 '22 22:09 DanJBower

(My statement above from 2 years ago is rather hyperbolic in hindsight I'm afraid. I discovered cases where hammett would fail in bad ways with the method that promised to be super fast)

I think that looks correct.

boxed avatar Sep 12 '22 07:09 boxed

You can run mutmut with the -s flag to see all the output. Maybe the hammett run ends up with zero tests?

boxed avatar Sep 12 '22 07:09 boxed

I just tried the -v to see the verbose output. It definitely detects the tests, but it's almost like it is caching the first mutant or something

I tried running it on two files, separately and together

1st file - it says it kills all the mutants (I was expecting this) 2nd file - it says it killed none (but I was expecting it to kill half) Both files at same time - it says it killed no mutants, even the ones against the first file which were working when run on its own

DanJBower avatar Sep 12 '22 10:09 DanJBower

@boxed I managed to reproduce the issue I was having on my system in a sample project. I've added it to a public repo

When running the batch file, you'll see the results:

Pytest as a runner - All these numbers are what I expected

  • src/sample/some_file.py: Killed 6/6
  • src/sample/some_other_file.py: Killed 6/10
  • Both: Killed 12 / 16

Hammet as a runner - It either seems to be all pass, or all fail. I was expecting the same results as using pytest as a runner

  • src/sample/some_file.py: Killed 6/6
  • src/sample/some_other_file.py: Killed 0/10
  • Both: Killed 16 / 16

https://github.com/DanJBower/HammettMutmutIssue

DanJBower avatar Sep 12 '22 12:09 DanJBower

@boxed By any chance were you able to reproduce the issue?

DanJBower avatar Nov 09 '22 11:11 DanJBower

Honestly I just don't have the time/energy to look at this. I will gladly accept a PR if you figure it out though.

boxed avatar Nov 09 '22 11:11 boxed

No worries

DanJBower avatar Nov 09 '22 13:11 DanJBower