Serge Smertin

[Feature]: Upgrade DBSQL Warehouses

7

Upstream dependencies: - https://github.com/databrickslabs/ucx/issues/670 TODO: - [ ] Set the default CATALOG to the new workspace catalog for DBSQL - [ ] Update all the warehouses to enable Unity Catalog...

Hive Metastore on multiple workspaces may point to the same assets. We need to dedupe upgrades.

7

### Is there an existing issue for this? - [X] I have searched the existing issues ### Problem statement Need to handle duplication of credentials & prefixes across different workspaces...

### Is there an existing issue for this? - [X] I have searched the existing issues ### Problem statement Redash queries can still use mounts ### Proposed Solution Compose checker...

Create a job to self-update the installation of the project

2

The job should: - look up github releases - get the latest tag - download release tarball (or wheel) - install the latest wheel on the job cluster (evaluate if...

If a view uses a function, we need to expose whether or not that function CAN be migrated

1

If a view uses a function, we need to expose whether or not that function CAN be migrated. _Originally posted by @zpappa in https://github.com/databrickslabs/ucx/pull/307#discussion_r1339110661_

Migration corner-case: what if the schema changed upstream in Hive. `SYNC` is supposed to be idempotent and intended to run multiple times.

2

What if the schema changed upstream in Hive. Sync is supposed to be idempotent and intended to run multiple times. _Originally posted by @zpappa in https://github.com/databrickslabs/ucx/pull/295#discussion_r1339133804_

- pay special attention to `cloudFiles` + bad records path

This PR adds `with_library(product, version)` method to clients & config, creating a copy with the user-agent extra propagated only for the relevant calls

Added `w.clusters.commands(cluster_id).run('return [1,2,3]')` utility

1

## Changes This PR refactors the wrapper around Command Execution API, which was used by `dbutils.fs.mounts()` and other cluster-proxied `dbutils` into a dedicated mixin. Usage: ```python cluster_id = env_or_skip("TEST_DEFAULT_CLUSTER_ID") ce...

Don't throw `TimeoutError` but rather the last error of retries

1

## Changes If we `raise TimeoutError(..) from last_err`, pytest displays the timeout error in the summary `FAILED ...::test_running_real_assessment_job - TimeoutError: Timed out after 0:06:00`, which makes troubleshooting harder during the...