Add flytekit-omegaconf plugin

[mg515]

Why are the changes needed?

Flytekit natively supports serialization of many data types for exchanging information between tasks.

The Flytekit OmegaConf Plugin extends these by the DictConfig type from the OmegaConf package as well as related types that are being used by the hydra package for configuration management.

Task example

from dataclasses import dataclass
from flytekit import task, workflow
from omegaconf import DictConfig
import flytekitplugins.omegaconf  # noqa F401


@dataclass
class MySimpleConf:
    _target_: str = "lightning_module.MyEncoderModule"
    learning_rate: float = 0.0001


@task
def my_task(cfg: DictConfig) -> None:
    print(f"Doing things with {cfg.learning_rate=}")


@workflow
def pipeline(cfg: DictConfig) -> None:
    my_task(cfg=cfg)


if __name__ == "__main__":
    from omegaconf import OmegaConf

    cfg = OmegaConf.structured(MySimpleConf)
    pipeline(cfg=cfg)

How was this patch tested?

Unit tests are added within the same PR.

Additionally, the plugin has been in private use for a while and is tested within multiple projects.

Check all the applicable boxes

• [x] I updated the documentation accordingly.
• [x] All new and existing tests passed.
• [x] All commits are signed-off.

Acknowledgements

The first version of the plugin was developed roughly at the end of year 2021 by some of the current and former members of Merantix Momentum. Shoutout to @SebS94 @fg91 @AdemFr @YmirKhang @axkoenig @adrianloy @ThomasWollmann

[welcome[bot]]

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

• Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
• Sign off your commits (Reference: DCO Guide).

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 45.99%. Comparing base (fa2aa0b) to head (a568a82). Report is 37 commits behind head on master.

:exclamation: Current head a568a82 differs from pull request most recent head 5abe25d. Consider uploading reports for the commit 5abe25d to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2299       +/-   ##
===========================================
- Coverage   83.89%   45.99%   -37.91%     
===========================================
  Files         342      199      -143     
  Lines       25483    19435     -6048     
  Branches     3725     3547      -178     
===========================================
- Hits        21380     8939    -12441     
- Misses       3472    10405     +6933     
+ Partials      631       91      -540     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mg515]

@SebS94 was having the same thoughts myself as well, could as well do it yeah.

[mg515]

Somehow the test builds started to fail after an innocent commit.

@pingsutw I saw that you tried to restart the checks already, but no success. Any idea what's going on?