[FEATURE]: Add the ability to configure and run a pipeline via CLI

[htahir1]

Contact Details [Optional]

[email protected]

Describe the feature you'd like

There are two ways to run a zenml pipeline:

• Hard-code the params in the runner script, or
• Configure the pipeline with a YAML config.

An example YAML might look like:

# config_template.yaml
steps:
  step_name:
    parameters:
      parameter_name: parameter_value
      some_other_parameter_name: 2
  some_other_step_name:
    ...

Concretely, a trainer step within a pipeline might look like:

# config.yaml
steps:
  trainer:
    parameters:
      lr: 0.005

And then run:

zenml pipeline run my_pipeline.py -c config.yaml

However, some times it might be nice to do:

zenml pipeline run my_pipeline.py --trainer --lr=0.005

I.e. directly via CLI. There needs to be clever grouping however.

Is your feature request related to a problem?

Sometimes it might be quick to just run a pipeline from the CLI without having to invoke python or create a YAML.

How do you solve your current problem with the current status-quo of ZenML?

Have to either hard-code params in the code or use the YAML config.

Any other comments?

No response

[htahir1]

A good place to start developing this feature is to see how the with_config feature works in code. This combined with inspecting the run_pipeline CLI command should be enough to get started

[schustmi]

Closing this as we're going in a different direction with this CLI call now that the ZenML server is able to register pipelines.