Run using a PipelineConfig class and CLI overrides

[lewfish]

To run a pipeline, you point the run CLI at a file with a get_config function that returns an instance of PipelineConfig. This is useful for storing particular configurations especially when they are very complex, but seems unnecessarily convoluted if you just want to run a pipeline using all the default options and a few overrides of those defaults. An alternative would be the ability to just point the run CLI at a PipelineConfig class, and supply any fields and their values to override. Example invocation: rastervision2 run my_module.MyPipelineConfig --override x 3 --override a.b "hello" This would make sense for pipelines that do some ephemeral data processing that was perhaps kicked off in an automated way by some other process.

Some things to think about:

• How to represent overrides of nested or array fields
• How to do input validation (can we harness the builtin Pydantic validation as much as possible)
• Generating useful --help docs that list the fields in an automated way from the Pydantic field descriptions

[lewfish]

@echeipesh originally proposed this idea

[lewfish]

See https://github.com/samuelcolvin/pydantic/issues/756 https://github.com/mpkocher/pydantic-cli