optimus icon indicating copy to clipboard operation
optimus copied to clipboard

move `defaultconfig` in plugins to `this.yaml`

Open smarchone opened this issue 3 years ago • 0 comments

Discussed in https://github.com/odpf/optimus/discussions/501

Originally posted by smarch-int August 8, 2022 Currently we are limited to override single plugin config using this.yaml. This limitation needs to be addressed by allowing multiple plugin configs to be merged into respective jobs.

Description Based on the above discussion, the static default configs in the plugins can be provided in the this.yaml spec in the project. They being static implies that they don't change at job level. This can be defined in this.yaml. This eliminates the need to explicitly define them in the plugins thereby simplifying the interfaces and codebase.

Acceptance Criteria

  • [ ] this.yaml to support config for multiple job types.
  • [ ] static config for the jobs should be inferred from this.yaml rather than plugins.
  • [ ] job execution should remain unaffected due to this change.

Tech Details

  • [ ] this.yaml to implement project spec rather than job spec.
  • [ ] Remove DefaultConfig from plugin interface and thereby all the binary implementations of plugins.
  • [ ] Infer default configs from this.yaml.

smarchone avatar Sep 14 '22 00:09 smarchone