Optionally disable creating output notebooks
Hi, I'm running papermill as part of a CI build and am getting this error:
Timeout waiting for IOPub output
I'm guessing that since the process has a lot of logging, the above error occurs when trying to write all of it to output notebooks.
It is possible to reduce the logging in this process, but the ideal fix for my case (assuming the output notebooks are the cause) would be to make creating output notebooks optional.
The output notebook is not the cause here. It's the kernel's ZMQ buffer overflowing when communicating back the outputs of execution. Instead you may want to set the log level for your CI (or as defaults for your notebook) to higher level and make sure the massive number of log messages are filterable on the source side of the equation. Also a few larger messages gets handled better than many small ones from a buffer overflow perspective.
Thanks, I'll give that a shot.