starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

Lake manual compaction failed to trigger jobs for all partitions on a table which has random generated partitions

Open wxl24life opened this issue 1 year ago • 0 comments

Steps to reproduce the behavior (Required)

create table t (id int) properties ("bucket_size"="16");

insert into t values (1);
insert into t values (2);
insert into t values (3);

show partitions from t;
+-------------+---------------+----------------+----------------+-------------+--------+--------------+-------+-----------------+---------+----------+----------+-----------------+------------+-------+-------+-------+
| PartitionId | PartitionName | CompactVersion | VisibleVersion | NextVersion | State  | PartitionKey | Range | DistributionKey | Buckets | DataSize | RowCount | EnableDataCache | AsyncWrite | AvgCS | P50CS | MaxCS |
+-------------+---------------+----------------+----------------+-------------+--------+--------------+-------+-----------------+---------+----------+----------+-----------------+------------+-------+-------+-------+
| 13014       | t             | 0              | 2              | 3           | NORMAL |              |       | ALL KEY         | 1       | 232B     | 1        | true            | false      | 0.00  | 0.00  | 0.00  |
| 13019       | t             | 0              | 2              | 3           | NORMAL |              |       | ALL KEY         | 1       | 232B     | 1        | true            | false      | 0.00  | 0.00  | 0.00  |
| 13006       | t             | 3              | 3              | 4           | NORMAL |              |       | ALL KEY         | 1       | 231B     | 1        | true            | false      | 0.00  | 0.00  | 0.00  |
+-------------+---------------+----------------+----------------+-------------+--------+--------------+-------+-----------------+---------+----------+----------+-----------------+------------+-------+-------+-------+
3 rows in set (0.006 sec)

Expected behavior (Required)

Triggering manual compaction on partition t should add 3 compactions jobs.

show proc '/compactions';
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
| Partition                                | TxnID | StartTime           | CommitTime          | FinishTime          | Error |
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
| sr_tpcds_bin_partitioned_orc_100.t.13014 | 5007  | 2024-02-27 12:49:53 | 2024-02-27 12:49:54 | 2024-02-27 12:49:54 | NULL  |
| sr_tpcds_bin_partitioned_orc_100.t.13006 | 5008  | 2024-02-27 12:49:53 | 2024-02-27 12:49:54 | 2024-02-27 12:49:54 | NULL  |
| sr_tpcds_bin_partitioned_orc_100.t.13019 | 5009  | 2024-02-27 12:49:53 | 2024-02-27 12:49:54 | 2024-02-27 12:49:54 | NULL  |
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
3 rows in set (0.004 sec)

Real behavior (Required)

Triggering manual compaction on partition t only add 1 compactions jobs.

show proc '/compactions';
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
| Partition                                | TxnID | StartTime           | CommitTime          | FinishTime          | Error |
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
| sr_tpcds_bin_partitioned_orc_100.t.13019 | 5009  | 2024-02-27 12:49:53 | 2024-02-27 12:49:54 | 2024-02-27 12:49:54 | NULL  |
+------------------------------------------+-------+---------------------+---------------------+---------------------+-------+
1 rows in set (0.004 sec)

StarRocks version (Required)

  • You can get the StarRocks version by executing SQL select current_version()
  • main branch

wxl24life avatar Feb 27 '24 04:02 wxl24life