cli
cli copied to clipboard
databricks
CI: Run tests without coverage in merge checks
Commit: https://github.com/databricks/cli/pull/3917/commits/4d53a50e06e239a247dbf44ed64f33e30d3b15e8
| Env | 🔄flaky | 💚RECOVERED | 🙈SKIP | ✅pass | 🙈skip | Time | |
|---|---|---|---|---|---|---|---|
| 🔄 | aws linux | 3 | 6 | 2 | 361 | 613 | 23:03 |
| 💚 | aws windows | 7 | 2 | 364 | 612 | 20:10 | |
| 💚 | aws-ucws linux | 7 | 2 | 501 | 503 | 28:37 | |
| 🔄 | aws-ucws windows | 3 | 4 | 2 | 502 | 502 | 25:35 |
| 💚 | azure linux | 1 | 4 | 363 | 612 | 20:15 | |
| 💚 | azure windows | 1 | 4 | 364 | 611 | 24:13 | |
| 💚 | azure-ucws linux | 1 | 4 | 497 | 502 | 39:00 | |
| 💚 | azure-ucws windows | 1 | 4 | 498 | 501 | 35:11 | |
| 🔄 | gcp linux | 4 | 1 | 4 | 355 | 615 | 24:58 |
| 🔄 | gcp windows | 3 | 4 | 358 | 614 | 22:50 |
17 failing tests:
| Test Name | aws linux | aws windows | aws-ucws linux | aws-ucws windows | azure linux | azure windows | azure-ucws linux | azure-ucws windows | gcp linux | gcp windows |
|---|---|---|---|---|---|---|---|---|---|---|
| TestAccept | 🔄f | 💚R | 💚R | 🔄f | 💚R | 💚R | 💚R | 💚R | 💚R | 🔄f |
| TestAccept/bundle/local_state_staleness | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
| TestAccept/bundle/local_state_staleness/DATABRICKS_BUNDLE_ENGINE=terraform | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
| TestAccept/bundle/resources/dashboards/change-name | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f |
| TestAccept/bundle/resources/dashboards/change-name/DATABRICKS_BUNDLE_ENGINE=terraform | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f |
| TestAccept/bundle/resources/dashboards/detect-change | 🔄f | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p |
| TestAccept/bundle/resources/dashboards/detect-change/DATABRICKS_BUNDLE_ENGINE=terraform | 🔄f | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p |
| TestAccept/bundle/resources/permissions | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions | 💚R | 💚R | 💚R | 🔄f | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct | 💚R | 💚R | 💚R | 🔄f | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions | 💚R | 💚R | 💚R | 💚R | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/run/app-with-job | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/templates/default-python/combinations/classic | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
| TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
Top 36 slowest tests (at least 2 minutes):
| duration | env | testname |
|---|---|---|
| 14:50 | azure windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 12:19 | azure windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 11:55 | azure windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 8:59 | azure-ucws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:55 | azure-ucws linux | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 8:31 | azure-ucws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 8:06 | azure-ucws windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 6:26 | gcp linux | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 6:06 | gcp windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 5:44 | gcp linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 5:38 | gcp windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 5:37 | gcp linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 5:21 | aws-ucws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 5:19 | aws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 5:18 | aws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 5:11 | aws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 5:09 | gcp windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 5:07 | aws-ucws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 4:58 | aws linux | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 4:58 | aws-ucws windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 4:25 | aws windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 3:55 | azure linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 3:52 | azure-ucws linux | TestAccept/bundle/resources/synced_database_tables/basic |
| 3:30 | azure-ucws windows | TestAccept/bundle/resources/synced_database_tables/basic |
| 3:29 | azure-ucws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 3:22 | azure linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 3:17 | azure-ucws windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 2:34 | gcp windows | TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9 |
| 2:31 | aws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 2:20 | aws-ucws linux | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 2:19 | azure-ucws linux | TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=yes |
| 2:17 | aws-ucws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 2:15 | aws-ucws linux | TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9 |
| 2:14 | azure-ucws linux | TestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform |
| 2:12 | aws-ucws linux | TestAccept/bundle/resources/models/basic/DATABRICKS_BUNDLE_ENGINE=terraform |
| 2:10 | aws-ucws linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
I changing this PR to only drop coverage testing from merge queue. The actual tests that are run are trickier - what's important there is what checks are expected by github, not what is actually run, so we need to configure that first.
Commit: https://github.com/databricks/cli/commit/562f59eff4659d2e8e260d150dd0e5ead8dea385
| Env | 🟨KNOWN | 🔄flaky | 💚RECOVERED | 🙈SKIP | ✅pass | 🙈skip | Time | |
|---|---|---|---|---|---|---|---|---|
| 🟨 | aws linux | 4 | 2 | 4 | 1 | 396 | 600 | 50:18 |
| 🟨 | aws windows | 4 | 6 | 1 | 397 | 599 | 53:09 | |
| 🟨 | aws-ucws linux | 4 | 6 | 1 | 552 | 484 | 87:07 | |
| 🟨 | aws-ucws windows | 4 | 6 | 1 | 553 | 483 | 71:47 | |
| 🟨 | azure linux | 3 | 1 | 3 | 396 | 599 | 52:34 | |
| 🟨 | azure windows | 3 | 1 | 3 | 397 | 598 | 47:01 | |
| 🟨 | azure-ucws linux | 3 | 3 | 3 | 546 | 483 | 72:55 | |
| 🟨 | azure-ucws windows | 4 | 3 | 549 | 482 | 63:07 | ||
| 🟨 | gcp linux | 4 | 2 | 3 | 381 | 605 | 78:49 | |
| 🟨 | gcp windows | 4 | 3 | 384 | 604 | 65:56 |
15 failing tests:
| Test Name | aws linux | aws windows | aws-ucws linux | aws-ucws windows | azure linux | azure windows | azure-ucws linux | azure-ucws windows | gcp linux | gcp windows |
|---|---|---|---|---|---|---|---|---|---|---|
| TestAccept | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K |
| TestAccept/bundle/integration_whl/interactive_single_user | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p | ✅p | ✅p |
| TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p | ✅p | ✅p |
| TestAccept/bundle/resources/permissions | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions | 💚R | 💚R | 💚R | 💚R | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions | 🔄f | 💚R | 💚R | 💚R | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S | 🙈S |
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct | 💚R | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform | 🔄f | 💚R | 💚R | 💚R | ||||||
| TestAccept/bundle/run/app-with-job | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K |
| TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct | 🟨K | 🟨K | 🟨K | 🟨K | 🔄f | 💚R | 🔄f | 🟨K | 🟨K | 🟨K |
| TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=terraform | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K | 🟨K |
| TestAccept/bundle/templates/default-python/integration_classic | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
| TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13 | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | ✅p | 🔄f | ✅p |
Top 50 slowest tests (at least 2 minutes):
| duration | env | testname |
|---|---|---|
| 15:22 | gcp linux | TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform |
| 12:36 | aws-ucws linux | TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform |
| 11:49 | aws-ucws linux | TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct |
| 11:34 | gcp windows | TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform |
| 11:34 | gcp linux | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION |
| 11:24 | aws-ucws windows | TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform |
| 11:22 | azure-ucws linux | TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform |
| 11:03 | azure-ucws windows | TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform |
| 10:53 | gcp windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER |
| 10:36 | aws-ucws windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER |
| 10:27 | gcp windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 10:26 | gcp windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 10:19 | gcp linux | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER |
| 10:15 | aws-ucws windows | TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct |
| 9:55 | gcp windows | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 9:52 | gcp windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform |
| 9:49 | gcp windows | TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=terraform |
| 9:35 | gcp windows | TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=direct |
| 9:18 | aws linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 9:07 | azure-ucws windows | TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:57 | azure windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:57 | aws linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:40 | azure windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 8:37 | azure linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:32 | aws-ucws windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION |
| 8:23 | aws windows | TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:14 | azure windows | TestAccept/bundle/integration_whl/custom_params/DATABRICKS_BUNDLE_ENGINE=direct |
| 8:06 | azure-ucws linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 8:00 | gcp windows | TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:59 | gcp windows | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:53 | aws-ucws linux | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION |
| 7:47 | aws windows | TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform |
| 7:44 | aws-ucws linux | TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_14.3_LTS |
| 7:42 | aws-ucws linux | TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_15.4_LTS |
| 7:40 | aws windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:39 | azure linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:36 | aws-ucws windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION |
| 7:34 | gcp linux | TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:32 | aws windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER |
| 7:32 | gcp linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 7:31 | aws-ucws windows | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER |
| 7:21 | aws-ucws windows | TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform |
| 7:21 | aws-ucws linux | TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:18 | aws-ucws linux | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:16 | aws-ucws windows | TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_13.3_LTS |
| 7:14 | azure-ucws windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform |
| 7:13 | azure-ucws windows | TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:11 | gcp linux | TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION |
| 7:07 | gcp linux | TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct |
| 7:06 | gcp linux | TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct |