ucx
ucx copied to clipboard
[FEATURE]: Table migration process only logs errors to files, but not to migration status tables for further analysis
Is there an existing issue for this?
- [X] I have searched the existing issues
Problem statement
When running migrate-tables
workflow, I want to know any failures, so that I can debug & re-run the process for those tables
Proposed Solution
Capture error during table migration process - potentially to a separate table
For example:
https://github.com/databrickslabs/ucx/blob/4ea2b0bcc783233913e75b555af5e5ef244354a5/src/databricks/labs/ucx/hive_metastore/table_migrate.py#L216-L224
logger.warning
is not enough, needs to also log the error to a ucx table, for further analysis
Additional Context
No response