projects icon indicating copy to clipboard operation
projects copied to clipboard

adding cookbook to pass variables from one task to the next one

Open edublancas opened this issue 2 years ago • 1 comments

A few times on Slack, we've been asked how to pass variables from one task to the next one. Typical use cases are runtime parameters (for example, doing feature selection and then training on that subset of features).

Our recommended solution is to serialize the parameters into a JSON file (or similar). However, this involves users learning how to use the JSON library. We should add a cookbook that shows how to do it, something like this:

tasks:
  # show an function task example
  - source: functions.some_function
     product:
       variable: products/variable.json

 # show a script/notebook example
   - source: scripts/some-script.py
     product:
       nb: products/report.html
       variable: products/another-variable.json

  # load  products/variable.json and products/another-variable.json her
   - source: scripts/last.py
      product: products/final.html

@idomic - tagging you to keep you in the loop. this isn't urgent, but if you need more good issues for the interns, this is a good one

edublancas avatar Jun 07 '22 19:06 edublancas

Created a few tags for us across the repos: high/med/low priority

idomic avatar Jun 07 '22 19:06 idomic