starrocks
starrocks copied to clipboard
[Enhancement] Introduce dataVersion, versionEpoch and versionTxnType to partition
Why I'm doing:
- Help cross-cluster replication achieve eventual consistency in failover.
- Help cross-cluster replication support do not disable compaction when replicating to a shared-data cluster.
What I'm doing:
Introduce dataVersion, versionSeq and versionTxnType to partition:
- dataVersion: dataVersion equals visibleVersion in shared-nothing mode, in shared-data mode, compactions increase visibleVersion but not dataVersion.
- versionEpoch: versionEpoch uses to check consistency of partitions in source and target cluster.
- versionTxnType: the type of the transaction to generate the version
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [ ] Yes, this PR will result in a change in behavior.
- [x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
[FE Incremental Coverage Report]
:white_check_mark: pass : 154 / 169 (91.12%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | com/starrocks/leader/ReportHandler.java | 0 | 1 | 00.00% | [1742] |
| :large_blue_circle: | com/starrocks/transaction/LakeTableTxnLogApplier.java | 6 | 9 | 66.67% | [54, 56, 91] |
| :large_blue_circle: | com/starrocks/transaction/OlapTableTxnLogApplier.java | 11 | 13 | 84.62% | [181, 189] |
| :large_blue_circle: | com/starrocks/catalog/Partition.java | 32 | 37 | 86.49% | [409, 410, 637, 640, 646] |
| :large_blue_circle: | com/starrocks/transaction/TransactionType.java | 13 | 15 | 86.67% | [46, 59] |
| :large_blue_circle: | com/starrocks/transaction/DatabaseTransactionMgr.java | 16 | 18 | 88.89% | [1277, 1299] |
| :large_blue_circle: | com/starrocks/transaction/OlapTableTxnStateListener.java | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/replication/ReplicationJob.java | 7 | 7 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/task/ClearTransactionTask.java | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/catalog/PhysicalPartitionImpl.java | 32 | 32 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/catalog/system/information/PartitionsMetaSystemTable.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/replication/ReplicationTxnCommitAttachment.java | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/common/proc/PartitionsProcDir.java | 14 | 14 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/transaction/LakeTableTxnStateListener.java | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/transaction/TransactionState.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/transaction/PartitionCommitInfo.java | 6 | 6 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/task/PublishVersionTask.java | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/service/InformationSchemaDataSource.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/lake/TxnInfoHelper.java | 1 | 1 | 100.00% | [] |
[BE Incremental Coverage Report]
:x: fail : 6 / 25 (24.00%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | be/src/exec/schema_scanner/schema_partitions_meta_scanner.cpp | 6 | 25 | 24.00% | [172, 174, 183, 189, 195, 201, 206, 211, 217, 228, 239, 244, 249, 255, 260, 265, 270, 275, 280] |
@Mergifyio backport branch-3.3
backport branch-3.3
✅ Backports have been created
- #49009 [Enhancement] Introduce dataVersion, versionEpoch and versionTxnType to partition (backport #46507) has been created for branch
branch-3.3but encountered conflicts
