Mateusz Pytel

Upgrade azure pipelines SDK to v2

3

Update Azure pipelines SDK to version 2. Reference docs - https://learn.microsoft.com/en-us/azure/machine-learning/migrate-to-v2-execution-pipeline?view=azureml-api-2

Current release candidate doesn't build on arm mac

1

``` Cannot install azureml-dataprep-native. - Installing azureml-dataprep-rslex (2.22.2): Failed RuntimeError Unable to find installation candidates for azureml-dataprep-rslex (2.22.2) ``` There are no wheels for Apple Silicon available for azureml v1,...

Extend E2E test scenario with mlflow support.

1

Current E2E tests don't cover use of kedro-mlflow. The idea is to extend tests coverage our own MlFlow instance.

Evaluate use of defaultdict for resources section handling in config.

1

What about using `from collections import defaultdict` with `resources: Optional[Dict[str, ResourcesConfig]] = defaultdict(lambda: ResourcesConfig(cpu="500m", memory="1024Mi"))` it would also simplify (or even make not needed) the `resources_for` function. _Originally posted by...

Solve FutureWarning: AIPlatformClient will be deprecated in v2.0.0

1

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/kfp/v2/google/client/client.py:169: FutureWarning: AIPlatformClient will be deprecated in v2.0.0. Please use PipelineJob https://googleapis.dev/python/aiplatform/latest/_modules/google/cloud/aiplatform/pipeline_jobs.html in Vertex SDK. Install the SDK using "pip install google-cloud-aiplatform" warnings.warn(

Allow to specify customer-managed encryption key for the Vertex pipeline execution.