Wrong YAML configuration file used

[nogitting]

Describe the issue

The CLI picks the wrong configuration file. What happens is, it starts from the working directory and goes up to the root for every supported configuration file, in this order: databricks.yml > databricks.yaml > bundle.yml > bundle.yaml, the first one found will be used. So, I reckon it makes sense to look for any of those files at every filesystem traversal iteration (i.e. searching by directory, not by configuration file) and/or to add support for a bundle_path option.

Steps to reproduce the behavior

1. Have a databricks.yml under /a.
2. Have a bundle.yml under /a/b/c.
3. From /a/b/c (as your PWD), try to run databricks bundle deploy ... for example.

Expected Behavior

bundle.yml should be used.

Actual Behavior

databricks.yml will be picked instead (and you will likely run into a no-such-target error).

OS and CLI version

macOS | v0.230.0

Is this a regression?

No.

Debug Logs

N/A

[pietern]

Thanks for creating the issue.

We allow the other filenames for backward compatibility with very old revisions, not because we recommend using those files. If you want the behavior you describe, I recommend using databricks.yml as an entry point everywhere and not mixing and matching different names.

Alternatively, you can configure the DATABRICKS_BUNDLE_ROOT environment variable to force the CLI to use the specified directory as the bundle root. The directory you specify must contain a valid entry point configuration file.

Could you elaborate on why you have a setup with nested bundles? (regardless of the entry point filename)

[nogitting]

We allow the other filenames for backward compatibility with very old revisions

Yes, even though my suggestion should take care of it without altering backward compatibility. (I am considering it an enhancement rather than a bug per se.)

I recommend using databricks.yml as an entry point everywhere and not mixing and matching different names.

We are definitely on the same page. However, the mix-and-match may happen (especially since the difference between .yml and .yaml matters), so ideally it should be safely caught or explicitly highlighted.

Alternatively, you can configure the DATABRICKS_BUNDLE_ROOT environment variable to force the CLI to use the specified directory as the bundle root

👍

Could you elaborate on why you have a setup with nested bundles? (regardless of the entry point filename)

A monorepo setup can increase the probability of this event.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.