doris icon indicating copy to clipboard operation
doris copied to clipboard

[opt](agg) Adjust the parameters of StreamingHtMinReductionEntry

Open mrhhsg opened this issue 2 weeks ago • 9 comments

What problem does this PR solve?

test with tpcds sf1000 Before: image

After: image

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • [ ] Regression test
    • [ ] Unit Test
    • [ ] Manual test (add detailed scripts or steps below)
    • [ ] No need to test or manual test. Explain why:
      • [ ] This is a refactor/code format and no logic has been changed.
      • [ ] Previous test can cover this change.
      • [ ] No code files have been changed.
      • [ ] Other reason
  • Behavior changed:

    • [ ] No.
    • [ ] Yes.
  • Does this need documentation?

    • [ ] No.
    • [ ] Yes.

Check List (For Reviewer who merge this PR)

  • [ ] Confirm the release note
  • [ ] Confirm test cases
  • [ ] Confirm document
  • [ ] Add branch pick label

mrhhsg avatar Dec 11 '25 03:12 mrhhsg

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Thearas avatar Dec 11 '25 03:12 Thearas

run buildall

morrySnow avatar Dec 11 '25 03:12 morrySnow

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Dec 11 '25 04:12 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Dec 11 '25 04:12 github-actions[bot]

TPC-H: Total hot run time: 34851 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 27f5c4673bf5d0521e5d9fae72455da041195465, data reload: false

------ Round 1 ----------------------------------
q1	17666	4256	4063	4063
q2	2003	350	249	249
q3	10204	1324	733	733
q4	10230	863	314	314
q5	7534	2075	1985	1985
q6	187	163	135	135
q7	1028	837	713	713
q8	9355	1459	1162	1162
q9	7278	5356	5308	5308
q10	6913	2383	1967	1967
q11	552	309	300	300
q12	682	699	595	595
q13	17789	3729	3051	3051
q14	313	285	274	274
q15	582	515	501	501
q16	685	678	619	619
q17	706	863	481	481
q18	7414	7132	6958	6958
q19	1100	958	621	621
q20	401	365	252	252
q21	4243	3979	3614	3614
q22	1083	1022	956	956
Total cold run time: 107948 ms
Total hot run time: 34851 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4289	4064	4029	4029
q2	336	413	321	321
q3	2167	2735	2315	2315
q4	1305	1741	1309	1309
q5	4249	4615	4780	4615
q6	214	174	131	131
q7	2025	1967	1893	1893
q8	2673	2512	2525	2512
q9	7638	7585	7490	7490
q10	3038	3254	2888	2888
q11	616	507	496	496
q12	737	874	622	622
q13	3558	3904	3432	3432
q14	303	305	291	291
q15	556	523	507	507
q16	629	654	668	654
q17	1240	1477	1435	1435
q18	7871	7615	7848	7615
q19	845	846	894	846
q20	1933	1946	1800	1800
q21	4574	4274	4180	4180
q22	1125	1033	982	982
Total cold run time: 51921 ms
Total hot run time: 50363 ms

doris-robot avatar Dec 11 '25 05:12 doris-robot

TPC-DS: Total hot run time: 178784 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 27f5c4673bf5d0521e5d9fae72455da041195465, data reload: false

query5	4668	619	465	465
query6	325	222	222	222
query7	4223	471	278	278
query8	308	260	248	248
query9	8773	2613	2607	2607
query10	528	386	332	332
query11	15355	14901	14885	14885
query12	189	116	116	116
query13	1263	510	423	423
query14	6198	3257	3010	3010
query14_1	2990	2968	2928	2928
query15	214	197	215	197
query16	907	459	429	429
query17	1109	697	580	580
query18	2568	425	336	336
query19	230	223	206	206
query20	117	109	115	109
query21	218	140	115	115
query22	3893	3924	3870	3870
query23	16557	16071	16013	16013
query23_1	16052	16079	15994	15994
query24	7390	1643	1247	1247
query24_1	1248	1212	1240	1212
query25	557	463	417	417
query26	1255	269	160	160
query27	2760	477	310	310
query28	4444	2152	2151	2151
query29	824	557	431	431
query30	316	248	220	220
query31	806	678	623	623
query32	73	73	67	67
query33	536	328	282	282
query34	898	897	543	543
query35	782	800	730	730
query36	879	915	851	851
query37	134	96	77	77
query38	2850	2797	2789	2789
query39	756	749	715	715
query39_1	700	693	704	693
query40	221	132	118	118
query41	65	62	61	61
query42	104	103	105	103
query43	423	429	392	392
query44	1320	760	753	753
query45	194	190	181	181
query46	870	962	626	626
query47	1659	1686	1607	1607
query48	313	331	245	245
query49	639	428	359	359
query50	677	298	216	216
query51	3889	3898	3798	3798
query52	103	107	96	96
query53	320	345	292	292
query54	322	258	270	258
query55	79	80	73	73
query56	282	294	313	294
query57	1147	1136	1080	1080
query58	275	252	250	250
query59	2358	2432	2271	2271
query60	323	310	284	284
query61	159	161	159	159
query62	694	661	633	633
query63	332	301	303	301
query64	4976	1296	989	989
query65	4016	3966	3946	3946
query66	1419	442	332	332
query67	14973	14835	14868	14835
query68	2760	1052	784	784
query69	453	352	333	333
query70	1031	978	1016	978
query71	353	307	293	293
query72	5834	4996	5005	4996
query73	470	537	315	315
query74	8846	8895	8677	8677
query75	3078	3102	2788	2788
query76	2842	1143	732	732
query77	346	398	291	291
query78	9592	9932	8858	8858
query79	1059	878	639	639
query80	1194	659	556	556
query81	552	267	240	240
query82	563	135	104	104
query83	359	247	247	247
query84	253	116	100	100
query85	909	497	453	453
query86	390	297	273	273
query87	3048	3086	2900	2900
query88	3365	2301	2273	2273
query89	476	425	391	391
query90	1990	158	158	158
query91	176	165	140	140
query92	67	67	66	66
query93	1183	918	569	569
query94	525	302	293	293
query95	559	329	300	300
query96	595	464	209	209
query97	2262	2279	2257	2257
query98	202	214	190	190
query99	1273	1306	1215	1215
Total cold run time: 252934 ms
Total hot run time: 178784 ms

doris-robot avatar Dec 11 '25 05:12 doris-robot

ClickBench: Total hot run time: 27.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 27f5c4673bf5d0521e5d9fae72455da041195465, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.28	0.24	0.25
query6	1.16	0.63	0.62
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.57	0.50	0.51
query10	0.55	0.55	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.11
query13	0.63	0.59	0.60
query14	1.00	0.98	0.99
query15	0.81	0.80	0.79
query16	0.40	0.40	0.41
query17	1.02	1.01	0.98
query18	0.24	0.22	0.22
query19	1.87	1.85	1.83
query20	0.02	0.01	0.01
query21	15.43	0.31	0.14
query22	4.87	0.05	0.05
query23	15.99	0.28	0.10
query24	1.20	0.26	0.56
query25	0.07	0.05	0.06
query26	0.14	0.14	0.13
query27	0.06	0.08	0.05
query28	3.97	1.20	1.01
query29	12.59	3.98	3.18
query30	0.29	0.15	0.12
query31	2.82	0.61	0.39
query32	3.23	0.57	0.46
query33	2.94	3.00	3.00
query34	17.00	5.21	4.54
query35	4.61	4.54	4.57
query36	0.66	0.49	0.49
query37	0.11	0.06	0.06
query38	0.07	0.04	0.03
query39	0.05	0.04	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.39 s
Total hot run time: 27.14 s

doris-robot avatar Dec 11 '25 05:12 doris-robot

BE UT Coverage Report

Increment line coverage :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 53.36% (18764/35166)
Line Coverage 39.11% (173520/443691)
Region Coverage 33.77% (134493/398206)
Branch Coverage 34.71% (57850/166643)

doris-robot avatar Dec 11 '25 07:12 doris-robot

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 72.28% (24914/34468)
Line Coverage 59.02% (261592/443215)
Region Coverage 53.91% (217234/402929)
Branch Coverage 55.45% (92887/167518)

hello-stephen avatar Dec 11 '25 08:12 hello-stephen