[Feature]: Add `Expirating` tab for table detail page on Dashboard

[zhoujinsong]

Description

Just like the Optimizing tab, we should add an Expiratiing tab in the table detail page to display all historical and ongoing expiration operations on the table, including snapshot expiration, orphan file cleaning, data file expiration, etc.

Use case/motivation

No response

Describe the solution

No response

Subtasks

No response

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[xxubai]

Hi @zhoujinsong , I would like to handle this feature. Can you split this feature into multiple subtasks first?

[xxubai]

Here is a summary of design about how to persist Expiring history:

Entity

In order to display records on the AMS dashboard, the expiration history needs to be persisted in the sysdb. The following MySQL DDL statement illustrates the table's entity:

CREATE TABLE `table_expiring_process`
(
    `process_id`                    bigint(20) NOT NULL COMMENT 'expiring_procedure UUID',
    `table_id`                      bigint(20) NOT NULL,
    `catalog_name`                  varchar(64) NOT NULL COMMENT 'Catalog name',
    `db_name`                       varchar(128) NOT NULL COMMENT 'Database name',
    `table_name`                    varchar(128) NOT NULL COMMENT 'Table name',
    `expiring_type`                 varchar(16) NOT NULL COMMENT 'Expiring type: ORPHAN_FILE, DANGLING_FILE, SNAPSHOT, DATA',
    `expire_time`                   timestamp DEFAULT CURRENT_TIMESTAMP COMMENT 'Starting expiring timestamp',
    `summary`                       varchar(512) COMMENT 'Summary of clean-up, such as count of total files',
    `file_paths`                    mediumtext COMMENT 'List of expired files',
    PRIMARY KEY (`process_id`),
    KEY  `table_index` (`table_id`, `expire_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT 'Expiration history of files in tables';

Except table entity, we also need a class to save the expiring information: ExpiringProcess

Mechanism

IcebergTableMaintainer and MixedTableMaintainer incorporate relevant metrics into ExpiringProcess during their operations(eg. cleanOrphanFiles, expireSnapshots, expireData). MixedTableMaintainer is responsible for gathering expiration history metrics from both BaseTableMaintainer and ChangeTableMaintainer. Once the collection is complete, ExpiringProcess will persist the expiration records into sysdb.

[xxubai]

WDYT @zhoujinsong , @wangtaohz

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'