papermill icon indicating copy to clipboard operation
papermill copied to clipboard

Executing a notebook with parameters, but no expected output

Open tgandor opened this issue 5 years ago • 2 comments

Is there a possiblity to run a notebook, without either changing it's original "template" (the notebook to process) or producing any output notebook. I.e. just take advantage of the parametrization, and the actual results are produced as side-effects, not as notebook content?

There is a simple, yet not portable way to do it:

$ papermill template.ipynb - -p value 5 > /dev/null

or

C:\> papermill template.ipynb - -p value 5 > NUL

Would it be possible to have a commandline option like --no-output which prevents any output creation, possibly enabling some optimization later on?

tgandor avatar Nov 18 '19 16:11 tgandor

I see it was discussed here already: https://github.com/nteract/papermill/issues/405

Redirecting to /dev/null, or NUL in particular has some downsides - for example when I do something like:

C:\> my_wrapper.bat papermill template.ipynb - > NUL

will also kill my wrapper's output. However, passing the redirection like that -> (arrow) creates a nice symbolism. Almost like the famous "goes towards" --> operator in while(n --> 0) statement.

When I go with MSeals' advice:

C:\> my_wrapper.bat papermill template.ipynb NUL

the wrapper output is fine, but I get:

...\site-packages\papermill\iorw.py:115: 
UserWarning: the file is not specified with any extension : NUL
  "the file is not specified with any extension : " + os.path.basename(path)

which is not nice, if you don't like warnings.

tgandor avatar Nov 19 '19 11:11 tgandor

@tgandor You can set the --log-level ERROR to suppress warnings. You may want to consider however that you're erasing your execution logs by not logging or saving the notebook result. I'd suggest doing --log-output and saving your logs somewhere and/or saving the notebook output to a temp directory so you can lookup what actually happened in the future.

MSeal avatar Nov 19 '19 18:11 MSeal