[Feature] support autonomous lake compaction compaction framework

[meegoo]

Why I'm doing:

What I'm doing:

Fixes #issue

What type of PR is this:

• [ ] BugFix
• [x] Feature
• [ ] Enhancement
• [ ] Refactor
• [ ] UT
• [ ] Doc
• [ ] Tool

Does this PR entail a change in behavior?

• [ ] Yes, this PR will result in a change in behavior.
• [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
• [ ] Parameter changes: default values, similar parameters but with different default values
• [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
• [ ] Feature removed
• [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• [x] I have added test cases for my bug fix or my new feature
• [ ] This pr needs user documentation (for new or modified features or behaviors)
  • [ ] I have added documentation for my new feature or new function
• [ ] This is a backport pr

Bugfix cherry-pick branch check:

• [x] I have checked the version labels which the pr will be auto-backported to the target branch
  • [ ] 4.0
  • [ ] 3.5
  • [ ] 3.4
  • [ ] 3.3

---

[!NOTE] Introduce autonomous lake compaction (async BE-scheduled, recoverable via local results) with FE/BE wiring, proto/config updates, partial publish support, and per-tablet parallel compaction hooks.

• Backend (BE):
  • Add lake/AutonomousCompactionHandler, lake/CompactionResultManager (local result scan/validate/save/cleanup), and lake/LakeCompactionManager (score-based queue, per-tablet concurrent tasks, rowset exclusion, size limits).
  • Extend lake/CompactionPolicy with pick_rowsets_with_limit and partial-compaction selector helpers.
  • Enhance lake/CompactionScheduler to accept autonomous mode and per-tablet parallel compaction; integrate TabletParallelCompactionManager and propagate table/partition/visible_version.
  • Add TabletManager::compact(context, input_rowsets) overload for preselected rowsets.
  • Wire-ups and builds (CMakeLists.txt).
• Frontend (FE):
  • CompactionScheduler: populate new fields (tableId, partitionId, visibleVersion), enable autonomous mode and optional per-tablet parallel config in requests.
  • Add system table id constant BE_TABLET_WRITE_LOG_ID.
• Proto:
  • CompactRequest: add table_id, partition_id, autonomous_compaction, visible_version, parallel_config.
  • New TabletParallelConfig; CompactResponse adds subtask_statuses.
  • New CompactionResultPB in lake_types.proto.
• Config:
  • Introduce multiple BE/FE configs for autonomous compaction, recovery, thresholds, and per-tablet parallel compaction.
• Tests:
  • Add UTs for AutonomousCompactionHandler and CompactionResultManager; update test build lists.

^{Written by Cursor Bugbot for commit c448701f75a870962ebbb2cb00dc5e90e7a8d62f. This will update automatically on new commits. Configure here.}

[mergify[bot]]

🧪 CI Insights

Here's what we observed from your CI run for c448701f.

🟢 All jobs passed!

But CI Insights is watching 👀

[github-actions[bot]]

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

[github-actions[bot]]

[FE Incremental Coverage Report]

:x: fail : 0 / 12 (00.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
:large_blue_circle:	com/starrocks/lake/compaction/CompactionScheduler.java	0	12	00.00%	[410, 411, 415, 416, 417, 418, 419, 420, 421, 454, 462, 463]

[github-actions[bot]]

[BE Incremental Coverage Report]

:white_check_mark: pass : 30 / 34 (88.24%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
:large_blue_circle:	src/storage/lake/compaction_policy.cpp	26	30	86.67%	[505, 506, 507, 515]
:large_blue_circle:	src/storage/lake/compaction_policy.h	2	2	100.00%	[]
:large_blue_circle:	src/storage/lake/compaction_scheduler.cpp	2	2	100.00%	[]

[alvin-celerdata]

@cursor review