Michael Schuster

Separate step configurations from deployment

1

## Describe changes This PR tackles some issues with pipeline deployments: - Adds an option to specify which attributes to skip during dehydration. This is used to avoid recursing through...

Improved Kubernetes orchestrator pod caching

2

## Describe changes This PR - enables orchestrator pod caching for scheduled runs. - delays the caching decision to run right before the actual step pod starts, not at orchestrator...

Split run submission and monitoring

2

## Describe changes This PR introduces a new method `submit_pipeline(...)` on the orchestrator interface as a replacement for the `prepare_or_run_pipeline(...)` method. For most orchestrators, the `prepare_or_run_pipeline` method had two main...

Dynamically import entrypoint in root __init__

2

## Describe changes The `entrypoint` module in the root `__init__` is now imported dynamically only when accessed, which fixes the following runtime warning `RuntimeWarning: 'zenml.entrypoints.entrypoint' found in sys.modules after import...

Switch from pkg_resources to importlib

1

## Describe changes This PR replaces all remaining uses of the deprecated and soon to be removed `pkg_resources` package with `importlib`. ## Pre-requisites Please ensure you have done the following:...

Better handling of committing nested transactions

1

## Describe changes I implemented/fixed _ to achieve _. ## Pre-requisites Please ensure you have done the following: - [ ] I have read the **CONTRIBUTING.md** document. - [ ]...

Runtime environment variables specification

2

## Describe changes This PR enables defining secrets and environment variables which will be set in the environment when executing steps of a pipeline. - Precedence: `Component < Stack <...

Allow returning existing artifacts from steps

1

## Describe changes This PR allows users to return an artifact version that is registered in the ZenML server from a step. Our caching behaviour currently uses the artifact version...

POC: Cache step while running

1

## Describe changes ```python @step(enable_cache=False) def step_with_potential_runtime_caching() -> int: should_cache = True # do some computation here if should_cache: return CachedStepResult() else: return 123 ``` ## Pre-requisites Please ensure you...

## Describe changes This PR enables the `oneof` operator when trying to filter some entity by a referenced entity (e.g. filter pipeline runs by the stack that they were run...