starrocks
starrocks copied to clipboard
[Test][Draft] DR test
Why I'm doing:
What I'm doing:
Fixes #issue
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [ ] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [ ] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [ ] I have checked the version labels which the pr will be auto-backported to the target branch
- [ ] 4.0
- [ ] 3.5
- [ ] 3.4
- [ ] 3.3
[!NOTE] Adds partition snapshot task and file sync pipelines with new RPCs/thread pools on BE and a FE job framework for automated cluster snapshots, plus related configs, metrics, and API updates.
- Backend (BE):
- Add
PARTITION_SNAPSHOTtask support: agent wiring, handlerrun_partition_snapshot_task, dedicated thread pool, shutdown handling, and metrics.- Implement lake partition snapshot execution (
storage/lake/partition_snapshot_task*) that computes file diffs and invokes new RPC to upload files.- Add
LakeService.upload_snapshot_filesRPC andSnapshotFileSyncerto copy snapshot data files; createsnapshot_file_synerthread pool inExecEnv.- New config
partition_snapshot_threads; expose corresponding metrics; addTBackend::operator<.- Enhance StarOS worker logging and FS building.
- Frontend (FE):
- Introduce snapshot job framework:
ClusterSnapshotJob(abstract),FullClusterSnapshotJob,MetaClusterSnapshotJob, andSnapshotJobContext; refactor scheduler to use context and incremental state machine.- Wire finish handling for
PARTITION_SNAPSHOTinLeaderImplandClusterSnapshotMgr.- Add
ClusterSnapshotTask(thriftTPartitionSnapshotRequest) and batching inAgentBatchTask.- Extend
StarOSAgentto allocate generic file paths and create virtual-tablet shard/group; persist vTablet info inStorageVolume.- Tests for cluster snapshot flow and full snapshot job.
- IDL / Protocols:
- Thrift: add
TTaskType::PARTITION_SNAPSHOT,TPartitionSnapshotRequest,TPartitionSnapshotInfoand wire into agent/master APIs.- Proto: add
UploadSnapshotFiles{Request,Response}PBand service RPC; defineTabletDataSnapshotPB.Written by Cursor Bugbot for commit 70034c6bcc1bf2eed5745edb785e63f5290ff297. This will update automatically on new commits. Configure here.
🧪 CI Insights
Here's what we observed from your CI run for c3eb2219.
🟢 All jobs passed!
But CI Insights is watching 👀
@cursor review