ucx icon indicating copy to clipboard operation
ucx copied to clipboard

[FEATURE]: Table migration process only logs errors to files, but not to migration status tables for further analysis

Open nkvuong opened this issue 9 months ago • 1 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Problem statement

When running migrate-tables workflow, I want to know any failures, so that I can debug & re-run the process for those tables

Proposed Solution

Capture error during table migration process - potentially to a separate table For example: https://github.com/databrickslabs/ucx/blob/4ea2b0bcc783233913e75b555af5e5ef244354a5/src/databricks/labs/ucx/hive_metastore/table_migrate.py#L216-L224 logger.warning is not enough, needs to also log the error to a ucx table, for further analysis

Additional Context

No response

nkvuong avatar May 23 '24 13:05 nkvuong