kedro icon indicating copy to clipboard operation
kedro copied to clipboard

Flexible node selection in `kedro run` command line syntax

Open iwhalen opened this issue 1 year ago • 3 comments

Description

I would like to be able to run nodes across namespaces in a more flexible way.

Context

Let's say I have three identical nodes across three namespaces: red.node, blue.node, yellow.node.

If I want to run them all, my only option is: kedro run --nodes red.node,blue.node,yellow.node

As number of namespaces increase this gets unweildy.

Possible Implementation

Add bash-style or dataset factory style wildcarding to the run command:

  • kedro run --nodes "*.node"
  • Or kedro run --nodes "{namespace}.node"

Possible Alternatives

Suggestion from @datajoely:

DBT style syntax for inclusion / exclusion / etc: https://docs.getdbt.com/reference/node-selection/syntax

iwhalen avatar Jun 12 '24 12:06 iwhalen

Related - https://github.com/kedro-org/kedro/issues/2552

datajoely avatar Jun 12 '24 13:06 datajoely

Is this related too? #3679

astrojuanlu avatar Jun 12 '24 19:06 astrojuanlu

@ianwhale thank you for the issue submitted! We would like to add this ticket to our backlog and investigate how exactly we want to extend run command filtering interface given the related issues mentioned above.

ElenaKhaustova avatar Jul 29 '24 13:07 ElenaKhaustova