papermill
papermill copied to clipboard
Implement bash translator
Support translation of parameters to bash/*NIX shell
Thanks for papermill! I have used it for python notebooks, but a lot of my work is best done in shell, for which I use jupyter notebooks with the bash_kernel (via *.sh scripts and jupytext).
Alas papermill doesn't seem to support bash/shell script, due to a hitherto unimplemented bash translator. This PR implements a BashTranslator
, and therefore enables bash kernel notebooks to be parametrised. This code supports str
, list
, bool
[^1], None
, int
, and float
parameter values. Dict could in theory be supported via associative arrays but for now they are not implemented[^2].
For a functioning example, please see the attached notebook.
Best, K
[^1]: Bash doesn't really have a bool type, but true
and false
are at least common and sensible string constants for truthy-ness and falsy-ness. See for example this SO answer.
[^2]: Associative arrays require quite recent bash versions, and I've never seen them used outside tutorials. Supporting them would also likely require a more involved code path as one must pre-declare an associative array in bash, and then declare one entry per line (see link above).
Alas GH doesn't support attaching *.ipynb, so I have had to rename it *.txt, but here's the test notebook.
The following params.yml can be used with this:
say: "Write a book"
people:
- Alice
- Bob
- Robert'); DROP TABLE students;-- # XKCD's favourite son
A friendly ping to the nteract team about this PR. All tests pass locally, and this is currently blocking us from using papermill in our analyses.
@rohitsanj tests implemented and passing. There are some unrelated failures on my machine, all to do with the google cloud interaction, which I assume is because I don't have a GCS environment correctly configured. Any chance you can allow the CI to run on this PR, so we can see if all is green in a well-configured environment?
Codecov Report
Merging #674 (000a80a) into main (98013f0) will increase coverage by
0.13%
. The diff coverage is90.90%
.
@@ Coverage Diff @@
## main #674 +/- ##
==========================================
+ Coverage 91.72% 91.86% +0.13%
==========================================
Files 17 17
Lines 1583 1622 +39
==========================================
+ Hits 1452 1490 +38
- Misses 131 132 +1
Impacted Files | Coverage Δ | |
---|---|---|
papermill/translators.py | 98.38% <90.90%> (-0.48%) |
:arrow_down: |
papermill/parameterize.py | 97.91% <0.00%> (-0.05%) |
:arrow_down: |
papermill/execute.py | 100.00% <0.00%> (ø) |
|
papermill/engines.py | 98.37% <0.00%> (+0.09%) |
:arrow_up: |
papermill/iorw.py | 81.15% <0.00%> (+0.91%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 98013f0...000a80a. Read the comment docs.