cli

cli copied to clipboard

Describe the issue

If you copy email notification section from a successfully deployed job, it will look like this(switched to variables for email address):

      email_notifications:
        on_failure:
          - ["${var.ALERT_EMAILS}"]
        on_duration_warning_threshold_exceeded:
          - ["${var.ALERT_EMAILS}"]

However this won't work as the email won't show up in the deployed job. You have to write this way:

          email_notifications:
            on_failure: ["${var.ALERT_EMAILS}"]
            on_duration_warning_threshold_exceeded: ["${var.ALERT_EMAILS}"]

Configuration

Please provide a minimal reproducible configuration for the issue

Steps to reproduce the behavior

Please list the steps required to reproduce the issue, for example:

1. Run databricks bundle deploy ...
2. Run databricks bundle run ...
3. See error

Expected Behavior

Normal job deployment with email shows up

Actual Behavior

No emails shows up

OS and CLI version

Please provide the version of the CLI (eg: v0.1.2) and the operating system (eg: windows). You can run databricks --version to get the version of your Databricks CLI MacOS, Databricks Cli 0.215

Is this a regression?

Debug Logs

Output logs if you run the command with debug logs enabled. Example: databricks bundle deploy --log-level=debug. Redact if needed

Thanks for reporting.

I'm unable to reproduce the issue. If I create a job with multiple entries for notification, I get the following:

          email_notifications:
            on_failure:
              - [email protected]
              - [email protected]

This is valid.

Are you copying from the web YAML viewer for a job as well, or have a different mechanism to get to the incorrect value?

Thanks for reporting.

I'm unable to reproduce the issue. If I create a job with multiple entries for notification, I get the following:

          email_notifications:
            on_failure:
              - [email protected]
              - [email protected]

This is valid.

Are you copying from the web YAML viewer for a job as well, or have a different mechanism to get to the incorrect value? @pietern Can you share your testing code? In the job yaml you should pass the arguments like this:

      email_notifications:
        on_failure:
          - ["${var.ALERT_EMAILS}"]
        on_duration_warning_threshold_exceeded:
          - ["${var.ALERT_EMAILS}"]

to replicate

I got the above working snippet by going into the web UI and navigating to View YAML/JSON:

Where did you find the snippet that you're reporting?

@pietern We were previously using dbx, and the code was like this:

				"email_notifications": {
					"on_start": [],
					"on_success": [],
					"on_failure": [{{ environ('ALERT_EMAILS') or ALERT_EMAILS}}],
					"on_duration_warning_threshold_exceeded": [{{ environ('ALERT_EMAILS') or ALERT_EMAILS}}]
				}

When migrating to databricks asset bundles. I used:

      email_notifications:
        on_failure:
          - ["${var.ALERT_EMAILS}"]
        on_duration_warning_threshold_exceeded:
          - ["${var.ALERT_EMAILS}"]

Because it seems intuitive: you keep the square bracket and use a variable. To my surprise databricks doesn't give me an error or anything to stop sqaure bracket usage. But it seems it can only handle email correctly with square bracket when the email and sqaure bracket is on the same line as the keyword.

Ah, I see.

The square brackets delineate a sequence in YAML. The leading - also implies a sequence, so you end up with a nested sequence. Newer versions of the CLI produce a warning if you use the configuration you provided and run databricks bundle validate:

Warning: expected string, found sequence
  at resources.jobs.default_python_job.tasks[1].email_notifications.on_failure[0]
  in resources/default_python_job.yml:30:17

Warning: expected string, found sequence
  at resources.jobs.default_python_job.tasks[1].email_notifications.on_duration_warning_threshold_exceeded[0]
  in resources/default_python_job.yml:32:17

Closing the issue as this is working as intended.