kedro icon indicating copy to clipboard operation
kedro copied to clipboard

Make users aware that arguments provided through CLI flags (--name, --tools, --example) overwrite arguments provided through --config

Open AhdraMeraliQB opened this issue 1 year ago • 2 comments

Description

Running kedro new --name=test --config=config.yml with the following config:

"project_name": 
    "My Project2"

"repo_name": 
    "my-project2"
            
"python_package":
    "my project2"

"tools":
    "none"

"example_pipeline":
    "no"

results in the following output:

Congratulations!
Your project 'test' has been created in the directory 
<path to directory>/my-project2

You have selected no project tools

To skip the interactive flow you can run `kedro new` with
kedro new --name=<your-project-name> --tools=<your-project-tools> --example=<yes/no>

Expected Result

An error when both are supplied, or a clear message that one is being overwritten

Actual Result

The --name argument silently overwrites the provided config

AhdraMeraliQB avatar Dec 18 '23 18:12 AhdraMeraliQB

Implementation-wise, CLI provided arguments will always overwrite any arguments provided through config. However, this seems to be less of a bug and more of an intentional implementation. Updating the title of this issue to address this, and instead the suggested actions are:

  • [ ] Add a log at warning level to alert users when they overwrite their configuration
  • [ ] Make it clear in the new project documentation that CLI flags would overwrite arguments in config

AhdraMeraliQB avatar Mar 25 '24 13:03 AhdraMeraliQB

It was marked as a bug, which I don't agree with. CLI overwrite any existing configuration is expected and consistent with other CLI i.e. kedro run --params --config xxx.yml. I also think this is a rather niche case because I don't think anyone is using --config other than us.

It was marked as "medium" due to majority.

noklam avatar Mar 25 '24 15:03 noklam