dbt-databricks
dbt-databricks copied to clipboard
Update `_wait_for_cluster_to_start` func to check for libraries statuses
trafficstars
Resolves #821
Description
Enhance _wait_for_cluster_to_start to wait for library installation
Previously, the _wait_for_cluster_to_start function only checked if the cluster state was 'running'. This update adds a check to ensure all cluster libraries have been successfully installed (or skipped or restored) before proceeding. This ensures the cluster is fully ready for operations that depend on these libraries.
- Added
get_cluster_libraries_statusto fetch the status of cluster libraries. - Updated
_wait_for_cluster_to_startto wait until all libraries are in valid statuses ('INSTALLED', 'RESTORED', 'SKIPPED').
Checklist
- [x] I have run this code in development and it appears to resolve the stated issue
- [x] This PR includes tests, or tests are not required/relevant for this PR
- [ ] I have updated the
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.