[BugFix] Fix the bug of direct schema change
Why I'm doing:
If the segments of single rowset is overlapping, will should use heap merge when do direct schema change, otherwise the rows is not ordered by sort key.
How to reproduce:
CREATE TABLE `lineorder_1` (
`lo_orderkey` int(11) NOT NULL COMMENT "",
`lo_linenumber` int(11) NOT NULL COMMENT "",
`lo_custkey` int(11) NOT NULL COMMENT "",
`lo_partkey` int(11) NOT NULL COMMENT "",
`lo_suppkey` int(11) NOT NULL COMMENT "",
`lo_orderdate` int(11) NOT NULL COMMENT "",
`lo_orderpriority` varchar(16) NOT NULL COMMENT "",
`lo_shippriority` int(11) NOT NULL COMMENT "",
`lo_quantity` int(11) NOT NULL COMMENT "",
`lo_extendedprice` int(11) NOT NULL COMMENT "",
`lo_ordtotalprice` int(11) NOT NULL COMMENT "",
`lo_discount` int(11) NOT NULL COMMENT "",
`lo_revenue` int(11) NOT NULL COMMENT "",
`lo_supplycost` int(11) NOT NULL COMMENT "",
`lo_tax` int(11) NOT NULL COMMENT "",
`lo_commitdate` int(11) NOT NULL COMMENT "",
`lo_shipmode` varchar(11) NOT NULL COMMENT "",
INDEX t1 (`lo_partkey`) USING BITMAP COMMENT ''
) ENGINE=OLAP
DUPLICATE KEY(`lo_orderkey`, `lo_linenumber`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`lo_orderkey`) BUCKETS 1
PROPERTIES (
"compression" = "LZ4",
"fast_schema_evolution" = "true",
"replicated_storage" = "true",
"replication_num" = "1"
);
insert into lineorder_1 select * from lineorder limit 2000000;
select * from lineorder_1 where lo_orderkey>5427812 and lo_orderkey<=5927812;
select lo_orderkey from lineorder_1 where lo_orderkey>5427812 and lo_orderkey<=5927812;
The num of return rows is inconsistent.
What I'm doing:
Use heap merge for direct schema change if the segments of rwoset is overlapping.
What type of PR is this:
- [x] BugFix
- [ ] Feature
- [ ] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [ ] Yes, this PR will result in a change in behavior.
- [x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [ ] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.3
- [x] 3.2
- [x] 3.1
- [x] 3.0
- [x] 2.5
[FE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[BE Incremental Coverage Report]
:white_check_mark: pass : 99 / 114 (86.84%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | be/src/testutil/schema_test_helper.cpp | 35 | 50 | 70.00% | [18, 20, 22, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 38] |
| :large_blue_circle: | be/src/storage/schema_change.cpp | 9 | 9 | 100.00% | [] |
| :large_blue_circle: | be/src/testutil/tablet_test_helper.cpp | 54 | 54 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/task/engine_alter_tablet_task.cpp | 1 | 1 | 100.00% | [] |
@Mergifyio backport branch-3.3
@Mergifyio backport branch-3.2
@Mergifyio backport branch-3.1
@Mergifyio backport branch-3.0
@Mergifyio backport branch-2.5
backport branch-3.3
✅ Backports have been created
- #44937 [BugFix] Fix the bug of direct schema change (backport #44854) has been created for branch
branch-3.3
backport branch-3.2
✅ Backports have been created
- #44938 [BugFix] Fix the bug of direct schema change (backport #44854) has been created for branch
branch-3.2
backport branch-3.1
✅ Backports have been created
- #44939 [BugFix] Fix the bug of direct schema change (backport #44854) has been created for branch
branch-3.1but encountered conflicts
backport branch-3.0
✅ Backports have been created
- #44940 [BugFix] Fix the bug of direct schema change (backport #44854) has been created for branch
branch-3.0but encountered conflicts
backport branch-2.5
✅ Backports have been created
- #44941 [BugFix] Fix the bug of direct schema change (backport #44854) has been created for branch
branch-2.5but encountered conflicts