[DOC] Deprecate Job level retry

[junwei-db]

Affected Resource(s)

https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job#argument-reference.

Expected Details

According to official Databricks API: https://docs.databricks.com/api/workspace/jobs/create, there's no max_retries and retry_on_timeout setting at job level, only at task-level. Based on local experiments, applying max_retries or retry_on_timeout to a multi-task job is noop; applying to single-task job will set retries at the task level, not job level. We should remove them or mark them deprecated from the doc to avoid confusions to customers as there's no concept of job-level retry.

List of things to potentially add/remove:

This is a list of things to manipulate in the docs:

• [ ] Remove job-level max_retries and retry_on_timeout from the doc, or mark them deprecated.

Important Factoids

References

[twolffpiggott]

This was a big source of confusion for my team at least- we were making reference to the Terraform provider docs, and ostensibly successfully configuring retries at the job level, with no effect.

[junwei-db]

Reopen this issue because there's one more job-level parameter to deprecate from docs: min_retry_interval_millis: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job#argument-reference, which is also a task-level config rather than job-level config.