starrocks
starrocks copied to clipboard
[Enhancement] Support push down aggregate functions in mv rewrite
Why I'm doing:
For simple aggregate functions(eg: min/max/sum), we can push down the agg function even if it's not exactly matched:
// eg:
// sum(fn(col)) = fn(sum(col))
// min(fn(col)) = fn(min(col))
// max(fn(col)) = fn(max(col))
eg:
CREATE TABLE `tbl1` (
`k1` date,
`k2` decimal64(18, 2),
`k3` varchar(255),
`v1` bigint
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `k2`, `k3`)
DISTRIBUTED BY RANDOM
PROPERTIES (
"replication_num" = "1"
);
CREATE MATERIALIZED VIEW `mv1`
DISTRIBUTED BY RANDOM
REFRESH ASYNC
PROPERTIES (
"replication_num" = "1"
)
AS SELECT k1, k2, k3, sum(v1) from tbl1 group by k1, k2, k3
-- Cannot be rewritten by mv1!
select t1.k1,
sum(case when t1.k1 between date_add('2024-07-20', interval -1 month) and date_add('2024-07-20', interval 1 month) then t1.v1 else 0 end)
from tbl1 t1 group by t1.k1
What I'm doing:
- Supports to push down min/max/sum aggregate function
- If the argument is a column ref, and operator map contains the column ref, return true
- If the argument is a call operator and contains multi-column refs(Ony IF/CaseWhen is supported), ensure the aggregate column does not appear in the condition clause.
- Add more tests about
if(cond, val1, val2)andcase whenrewrite test cases.
Fixes #issue
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [ ] Yes, this PR will result in a change in behavior.
- [x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
@mergify rebase
rebase
✅ Branch has been successfully rebased
Quality Gate passed
Issues
18 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
[Java-Extensions Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[FE Incremental Coverage Report]
:white_check_mark: pass : 104 / 112 (92.86%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | com/starrocks/sql/optimizer/rule/transformation/materialization/EquationRewriter.java | 95 | 103 | 92.23% | [202, 263, 278, 286, 294, 300, 317, 336] |
| :large_blue_circle: | com/starrocks/sql/optimizer/rule/transformation/materialization/AggregateFunctionRollupUtils.java | 5 | 5 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/analyzer/QueryAnalyzer.java | 4 | 4 | 100.00% | [] |
[BE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)