nbval icon indicating copy to clipboard operation
nbval copied to clipboard

Reset magic breaks with coverage report

Open svaberg opened this issue 6 years ago • 2 comments

Hi all, I love your project!

I just noticed that the IPython %reset magic breaks the coverage report generated with

py.test -v --nbval-lax --cov .

I observe this even when it is only present in the first line of the first cell of the notebook. I made a small example notebook (attached) where the reported coverage varies between 0% and 100%.

This is with no %reset magic:

Name       Stmts   Miss  Cover
------------------------------
mylib.py       2      0   100%

and this is with one %reset magic in first code cell

Name       Stmts   Miss  Cover
------------------------------
mylib.py       2      2     0%

Do you think it would be possible to survive a %reset or must they be avoided?

svaberg avatar Dec 04 '19 00:12 svaberg

The coverage code relies on putting a variable in the global scope of the kernel (__cov). There might be a technical trick to do in nbval to ensure that would survive %reset, but I'm not sure. It might be possible to add some code in the notebook in question to avoid clearing this variable.

vidartf avatar Feb 12 '20 12:02 vidartf

Thanks @vidartf. Doing a "soft" reset %reset -s does not delete underscore and double underscore prepended variables, so I will use %reset -sf instead of %reset -f. This should work as long as the notebooks do not depend on underscore variables being erased by the reset command and is enough for my needs.

I think it would be possible to survive a %reset by redefining the magic method so that it copies __cov before the reset and puts it back afterwards, I might give that approach a try as well.

svaberg avatar Feb 19 '20 03:02 svaberg