kedro icon indicating copy to clipboard operation
kedro copied to clipboard

[DataCatalog]: Hard to to manage large catalogs

Open ElenaKhaustova opened this issue 8 months ago • 1 comments

Description

Users find it difficult to manage large catalogs as the current separate from code configuration structure requires excessive navigation back and forth, YAML-based data catalog is cumbersome to manage and navigate.

We propose to:

  1. Explore the opportunity to offer an alternative to YAML-based catalogs that can be integrated with the current configuration approach.
  2. Explore how existing VS Code plugin simplifies working with large catalogs and extend it with features for easy navigation.

Context

"I believe there's room for innovation in how the data catalog is structured in relation to the code. Currently, the configuration is organized differently and separately from the code, which requires a lot of navigation back and forth. Maybe an alternative where the catalog lives closer to where it's used could potentially reduce this overhead and improve productivity."

ElenaKhaustova avatar Jun 06 '24 15:06 ElenaKhaustova