feat(storage): support compaction group split and merge

[Li0k]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Previously, we implemented the concept of compaction groups for hummock to achieve write/configuration isolation and to create new compaction groups via the table-level move function.

Previously, we implemented the concept of compaction groups for hummock to achieve write/configuration isolation and to create new compaction groups via the table-level move function.

Since the sst key range generated by the move function is not contiguous, this makes it difficult to re-combine the resulting independent compaction groups (currently not supported). Because of this concern, we set the move threshold very high, fearing that the resulting compaciton group will not be able to be merged again, thus creating a large number of compaction groups.

This concern is contrary to our expectation that in scenarios with high write throughput (backfill, etc.), we would like to split the compaction to achieve parallel compaction, increase the overall compaction throughput, and make full use of the compactor node's scale capability.

Therefore, we redesigned hummock's scalability by implementing split / merge instead of move, in order to obtain the following capabilities.

1. adopt more lenient split conditions to cope with high write/burst write traffic scenarios, to extend compaction capability horizontally.
2. auto-merge compaction groups after write throughput is restored.
3. refine the granularity of Split from table to key_range, to split huge compaction groups at a finer granularity
4. Better O&M tools to manually split and merge compaction groups.

For the current PR, we expect to implement table-level Split/Merge capability (phase 1), and plan to further implement Key-level Split in the subsequent PRs, and make CN not aware of the concept of group_id. Here are the changes involved in this PR for review.

1. add the concept of estimated_size to SstableInfo for more accurate Compaction policy calculation.
2. based on table_id level Split function, compatible with the original Split interface.

• Fix key_range/table_id/estimated size after sst split.
• Use (table_id, vnode = 0, epoch = 0) as split key for splitting.

3. merge compaction group
4. schedule capability for automatic split/merge.
5. compactor iterator support key_range filter

Checklist

• [ ] I have written necessary rustdoc comments
• [ ] I have added necessary unit tests and integration tests
• [ ] I have added test labels as necessary. See details.
• [ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
• [ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• [x] All checks passed in ./risedev check (or alias, ./risedev c)
• [ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• [ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

• [ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.