paimon icon indicating copy to clipboard operation
paimon copied to clipboard
verified publisher icon apache

[Feature] Whole database compression combined mode does not support append unware bucket table

Open thomasg19930417 opened this issue 2 years ago • 4 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Motivation

Whole database compression combined mode does not support append unware bucket table。 f7467db14a7d8d34906f776d5695c4e3

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

thomasg19930417 avatar Jan 10 '24 09:01 thomasg19930417

assign to me plz

wg1026688210 avatar Jan 11 '24 07:01 wg1026688210

Hi~ @thomasg19930417 Can you provide the command ?

wg1026688210 avatar Jan 11 '24 08:01 wg1026688210

Hi~ @thomasg19930417 Can you provide the command ?

Use the entire database compression command and specify the compression mode --mode combined, and the table is an append table. There are no other special parameters.

thomasg19930417 avatar Feb 02 '24 09:02 thomasg19930417

I will fix this issue by breaking it down into several subtasks, which will be better to review.

  1. support specify database and table in AppendOnlyCompactionTask #3174
  2. support compact unaware bucket for combine_mode #3175
  3. refact the file scan source of combine mode for supporting unaware logic #3179
  4. use refacted combined compaction implement #3182

wg1026688210 avatar Apr 08 '24 13:04 wg1026688210