Should multi-notebook execution (and a few other features) be in-scope for this package?

[choldgraf]

We're working on another project that will likely involve some kind of multi-notebook output cacheing and execution. Things like:

• A class that can parallelize the execution of notebooks
• A way to check whether a notebook has become "stale" and should be re-executed to make sure outputs match the code cells or content
• A way to generate information about the execution status (stale, fresh, error states, etc) of multiple notebooks

I am trying to decide whether nbclient is the right home for these kinds of features, or if instead we should plan on depending on nbclient, but doing these features as part of another package. I'm happy w/ either approach, but curious what others think.

[MSeal]

Sorry for the late response.

A class that can parallelize the execution of notebooks

Possibly -- it can be tricky to generalize as there's multiple parallelization strategies. It might be better to leave the repo simple and not add parallelization opinions. I could be swayed on this, but I think that's the stance for more general repos.

A way to check whether a notebook has become "stale" and should be re-executed to make sure outputs match the code cells or content

I would say this definitely belong outside of nbclient, unless there was something added to the protocol to specifically support the idea for other jupyter projects.

A way to generate information about the execution status (stale, fresh, error states, etc) of multiple notebooks

Same as above. Though the doing https://github.com/jupyter/nbclient/issues/28 will give some of the information I think you'd use to inform staleness.

[choldgraf]

thanks - this is super useful feedback. we ended up piloting some prototypes for the "multi-notebook execution and cacheing" stuff here: https://github.com/executablebookproject/jupyter-cache

would love to know if you think that's interesting and/or useful! We will plan to keep this functionality in that package, but if there are ways to contribute back to nbclient in the process we are happy to make it happen