nbcelltests icon indicating copy to clipboard operation
nbcelltests copied to clipboard

Generate test notebooks on the fly rather than using actual, stored notebooks

Open ceball opened this issue 4 years ago • 3 comments

Currently, nbcelltests' tests use various .ipynb files stored in nbcelltests/tests. Instead, we should generate notebooks as needed during the tests.

Doing this will allow various bits of test code to be simplified.

ceball avatar May 27 '20 21:05 ceball

Hi I am interested doing this issue, but can we use other library to generate test notebooks or it is better to code it from scratch to generate jupyter notebook?

michiboo avatar May 28 '20 09:05 michiboo

Yes @michiboo thanks for checking - definitely better to use a library if that's possible.

I was imagining using something like nbformat's new_notebook(). E.g. here it is being used in nbval: https://github.com/computationalmodelling/nbval/blob/3597e0bfd96c8b52d9f4e94abbe1cbb3887dcf25/tests/utils.py#L5

I think I would probably start off by replacing the notebooks associated with https://github.com/jpmorganchase/nbcelltests/blob/master/nbcelltests/tests/test_lint.py and see how that works out.

Four notes I can think of right now:

  1. I'd like to leave a least one actual notebook file around, associated with a test that nbcelltests does actually work with a real file :)
  2. Just to warn you: a part of this task might be improving the tests themselves. There is probably some overlap between unit tests, partially because some re-use the same underlying notebook files. Ideally improving the tests would be a separate task (i.e. should be possible to do a straight up "generate the notebooks" replacement for "physical notebooks" only, but also possible that something blocks that).
  3. Testing tools that are themselves test tools can be tricky, so please feel free to ask if you see something that looks weird - it might be just bad code, or it might be necessary for some awkward reason.
  4. This task will leave you with a negative "line count contribution" to the project. That's the second best kind of contribution - second only to preventing lines being written in the first place :)

ceball avatar May 28 '20 12:05 ceball

Hi, I'd like to contribute on this issue, I am currently looking at the codebase and understand the working

sakshamk6999 avatar Jun 18 '20 10:06 sakshamk6999