nbval icon indicating copy to clipboard operation
nbval copied to clipboard

ENH: skip rest of file after failure

Open bsipocz opened this issue 5 months ago • 1 comments

This should address #145

~Still draft as it doesn't work along with rerunfailures, and I don't yet see why.~

TODO checklist:

  • [ ] add option for ini, or maybe also for cli
  • [x] make we don't skip the consecutive runs of rerunfailure
  • [ ] ensure that returned traceback is for the first cell failing; this doesn't seem to work when rerunfailure is also used
  • [ ] fixup tests (for unrelated ones I open separate PRs)
  • [ ] add docs

And also it will need an option to opt-in (or I would prefer if this would be opt-out, but that means breaking current behaviour).

================================================= test session starts ==================================================
platform darwin -- Python 3.12.1, pytest-8.4.0, pluggy-1.6.0
rootdir: /Users/bsipocz/munka/IPAC/navo-workshop
plugins: astropy-0.11.0, remotedata-0.4.1, anyio-4.3.0, custom-exit-code-0.3.0, filter-subpackage-0.2.0, doctestplus-1.4.0, rerunfailures-15.1, astropy-header-0.2.2, dependency-0.6.0, mock-3.14.1, darkgraylib-1.2.1, openfiles-0.6.0, nbval-1.11.0, cov-6.1.1, requests-mock-1.12.1
collected 8 items                                                                                                      

content/reference_notebooks/spectral_access.ipynb ...Fssss                                                       [100%]

...

=============================================== short test summary info ================================================
SKIPPED [4] ../../devel/other/nbval/nbval/plugin.py:943: Due to the previous failure skipping rest of tests in /Users/bsipocz/munka/IPAC/navo-workshop/content/reference_notebooks/spectral_access.ipynb
FAILED content/reference_notebooks/spectral_access.ipynb::Cell 3
======================================== 1 failed, 3 passed, 4 skipped in 7.71s ========================================

bsipocz avatar Jun 17 '25 02:06 bsipocz

While now the new commits fixes the compatibility with rerunfailures, the returned traceback is incorrect (one cell lower in the stack, and thus makes no sense; e.g. undefined variable).

So that adds a new TODO item for this PR, to track down if the error is due to this PR or due to rerunfailes.

bsipocz avatar Jun 17 '25 04:06 bsipocz