starrocks
starrocks copied to clipboard
[Enhancement] pushdown or predicate (4): support parse and evaluate or predicate
Why I'm doing:
Parse OR predicate
Parse OR expr to predciate tree.
Evaluate
Before:
- Iterate over each ColumnPredicate in
unordered_map<ColumnId, vector<ColumnPredicate*>>
, where the relationship between ColumnPredicates is AND.
After:
- Iterate over each ColumnPredicate in
PredicateTree
, where the relationship between ColumnPredicates is AND or OR.
What I'm doing:
Parse OR preidcate
If Type is OR, to convert OR to AND to utilize existing parsing logic, perform the Inverted
operation in normalize_xxx_predicate
and build_olap_filters
. For example, pred_c1_1 or pred_c1_2 or pred_c2_1 or pred_c2_2
will be converted to !(!pred_c1_1 and !pred_c1_2) or !(!pred_c2_1 and !pred_c2_2)
The specific steps are as follows:
- When normalizing predicates to value ranges, pass true to
<Inverted>
of eachnormalize_xxx_predicate
. - When building olap filters by the normalized value ranges, pass true to
<Inverted>
ofColumnValueRange::to_olap_filter
.
Evaluate
Introduce three methods evaluate
, evaluate_and
, evaluate_or
for each type of PredicateTreeNode.
Specially, the evaluate
method of PredicateTreeAndNode
will use evaluate_branchless
for non_vectorized ColumnPredicate
.
Relates to #43801.
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [ ] Yes, this PR will result in a change in behavior.
- [x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [ ] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
[FE Incremental Coverage Report]
:white_check_mark: pass : 7 / 7 (100.00%)
file detail
path | covered_line | new_line | coverage | not_covered_line_detail | |
---|---|---|---|---|---|
:large_blue_circle: | com/starrocks/qe/scheduler/TFragmentInstanceFactory.java | 3 | 3 | 100.00% | [] |
:large_blue_circle: | com/starrocks/qe/SessionVariable.java | 4 | 4 | 100.00% | [] |
[BE Incremental Coverage Report]
:white_check_mark: pass : 370 / 452 (81.86%)
file detail
path | covered_line | new_line | coverage | not_covered_line_detail | |
---|---|---|---|---|---|
:large_blue_circle: | be/src/connector/lake_connector.cpp | 0 | 30 | 00.00% | [78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 91, 92, 95, 134, 138, 245, 249, 340, 343, 345, 347, 348, 350, 365, 366, 544, 545, 546] |
:large_blue_circle: | be/src/exec/olap_scan_node.cpp | 0 | 15 | 00.00% | [537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 548, 549, 551, 671, 673] |
:large_blue_circle: | be/src/exec/tablet_scanner.cpp | 0 | 4 | 00.00% | [73, 152, 282, 286] |
:large_blue_circle: | be/src/exec/olap_utils.h | 4 | 8 | 50.00% | [123, 124, 125, 127] |
:large_blue_circle: | be/src/exec/pipeline/scan/olap_chunk_source.cpp | 7 | 11 | 63.64% | [449, 451, 623, 624] |
:large_blue_circle: | be/src/storage/predicate_tree/predicate_tree.cpp | 10 | 12 | 83.33% | [87, 88] |
:large_blue_circle: | be/src/storage/predicate_tree/predicate_tree.hpp | 80 | 94 | 85.11% | [166, 167, 251, 254, 255, 258, 266, 295, 296, 324, 344, 347, 348, 351] |
:large_blue_circle: | be/src/exec/olap_scan_prepare.cpp | 200 | 209 | 95.69% | [224, 242, 245, 309, 457, 991, 992, 994, 995] |
:large_blue_circle: | be/src/storage/predicate_tree/predicate_tree.h | 1 | 1 | 100.00% | [] |
:large_blue_circle: | be/src/exec/olap_scan_prepare.h | 1 | 1 | 100.00% | [] |
:large_blue_circle: | be/src/exec/pipeline/scan/olap_scan_context.cpp | 17 | 17 | 100.00% | [] |
:large_blue_circle: | be/src/exec/pipeline/fragment_executor.cpp | 3 | 3 | 100.00% | [] |
:large_blue_circle: | be/src/exec/pipeline/scan/olap_scan_context.h | 1 | 1 | 100.00% | [] |
:large_blue_circle: | be/src/exprs/expr.cpp | 1 | 1 | 100.00% | [] |
:large_blue_circle: | be/src/exec/olap_common.cpp | 12 | 12 | 100.00% | [] |
:large_blue_circle: | be/src/storage/rowset/segment_iterator.cpp | 33 | 33 | 100.00% | [] |