kedro icon indicating copy to clipboard operation
kedro copied to clipboard

[DataCatalog2.0]: Move config validation to the `CatalogConfigResolver` init

Open ElenaKhaustova opened this issue 5 months ago • 0 comments

Description

Currently, we resolve dataset configuration for patterns at the runtime. During the resolution, we validate the config format and raise the error in case of failure: https://github.com/kedro-org/kedro/blob/53280bd3ec188b7be3cba8834b4f000bc7fbbe91/kedro/io/catalog_config_resolver.py#L133

We suggest moving the config validation step to the CatalogConfigResolver init so that we do not allow creating a config resolver object with incorrect pattern configurations.

Context

https://github.com/kedro-org/kedro/pull/4151#discussion_r1773062101

Possible Implementation

  1. Move config validation step to the CatalogConfigResolver's constructor and raise an error in case an incorrect configuration is provided
  2. Simplify CatalogConfigResolver._resolve_dataset_config method keeping only filling placeholders logic

Possible Alternatives

ElenaKhaustova avatar Sep 24 '24 14:09 ElenaKhaustova