kedro-plugins icon indicating copy to clipboard operation
kedro-plugins copied to clipboard

Move kedro catalog validation schema to kedro-datasets

Open noklam opened this issue 1 year ago • 1 comments

Description

Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

https://github.com/kedro-org/kedro/blob/144323a05a708f0656f937858f8234386d6ac34f/static/jsonschema/kedro-catalog-0.19.json#L4

move kedro-catalog-0.19.json to kedro-datasets

Context

Why is this change important to you? How would you use it? How can it benefit other users? most dataset has moved to kedro-datasets already, it makes sense to version against kedro-datasets instead of kedro so the schema can support different versions of dataset easily.

Automation of: https://docs.kedro.org/en/stable/development/set_up_vscode.html#configuring-the-kedro-catalog-validation-schema

In addition, it's also easier for plugin to integrate with it. The json file should be packaged with the libraries, so pip install kedro-datasets should also download the data. I already did a PoC to include this in a VSCode extension, this would make future support much easier.

Possible Implementation

(Optional) Suggest an idea for implementing the addition or change.

  • Automatic update it via CI
  • Move that to kedro-datasets

Possible Alternatives

(Optional) Describe any alternative solutions or features you've considered.

noklam avatar Apr 21 '24 23:04 noklam

Explore if the schema validation works if we keep the core Kedro dataset (LambdaDataset) and move the remaining to kedro-datasets

noklam avatar Apr 22 '24 13:04 noklam