dbx

dbx copied to clipboard

Expected Behavior

Command launch and launch with --as-run-submit should be able to override parameters on multi-task job.

Current Behavior

Command launch and launch with --as-run-submit fail to override parameters on multi-task job.

Steps to Reproduce (for bugs)

For RunNow:

1. Create multi-task job/
2. Deploy.
3. Launch with parameter provided by --parameters or --parameters-raw keys. e.g.:

dbx launch \
  --job smaple \
  --environment default \
  --parameters-raw='{"key1": "value1", "key2": 2}' 

For RunSubmit:

1. Create multi-task job/
2. Deploy with --files-only.
3. Launch with parameter provided by --parameters or --parameters-raw keys. e.g.:

dbx launch \
  --job smaple \
  --environment default \
  --parameters-raw='{"key1": "value1", "key2": 2}' \
  --as-run-submit

Context

Your Environment

• dbx version used: v0.3.0
• Databricks Runtime version: 10.1

hey @Vakha , It took some time for me to review your PR and it sparked some interest in a more profound refactoring of the whole project. I'm currently working on it, hopefully shall finish by the EOW. Sorry for delays!

Can we expect this anytime soon?

hi @HiteshDream , multitask jobs are already supported, but parameter passing via CLI is not yet fixed. Please take a look at --task parameter in latest version.

Hi @renardeinside I have a requirement to run my tasks via cli on all purpose cluster (via dbx execute), hence I need a way to pass parameters to my task via CLI. Is there any way to do that? If not is there any plans to add this support?

hi @HiteshDream , this functionality will be added in 0.7.0, currently in heavy development. Stay tuned, we shall be able to release it in 1 week