doris icon indicating copy to clipboard operation
doris copied to clipboard

[opt](nereids) add join shuffle type in explain shape plan

Open englefly opened this issue 1 year ago • 70 comments

Proposed changes

in previous version, users need to determine join shuffle type by join children mannually. It is error-prone. we print shuffle type in explain shape plan to improve plan readability Issue Number: close #xxx

englefly avatar Jun 23 '24 08:06 englefly

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar Jun 23 '24 08:06 doris-robot

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In post_build.sh line 1:
cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.

For more information:
  https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and y...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- post_build.sh.orig
+++ post_build.sh
@@ -1,4 +1,4 @@
-cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
+cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids fe/fe-core/target/generated-sources/org/apache/doris/
 cp -r fe/fe-core/target/generated-sources/cup/* fe/fe-core/target/generated-sources/
 rm -rf fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids
 rm -rf fe/fe-core/target/generated-sources/cup/
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


github-actions[bot] avatar Jun 23 '24 08:06 github-actions[bot]

run buildall

englefly avatar Jun 23 '24 08:06 englefly

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In post_build.sh line 1:
cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.

For more information:
  https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and y...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- post_build.sh.orig
+++ post_build.sh
@@ -1,4 +1,4 @@
-cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
+cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids fe/fe-core/target/generated-sources/org/apache/doris/
 cp -r fe/fe-core/target/generated-sources/cup/* fe/fe-core/target/generated-sources/
 rm -rf fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids
 rm -rf fe/fe-core/target/generated-sources/cup/
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


github-actions[bot] avatar Jun 23 '24 08:06 github-actions[bot]

run buildall

englefly avatar Jun 23 '24 08:06 englefly

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In post_build.sh line 1:
cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.

For more information:
  https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and y...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- post_build.sh.orig
+++ post_build.sh
@@ -1,4 +1,4 @@
-cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids  fe/fe-core/target/generated-sources/org/apache/doris/
+cp -r fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids fe/fe-core/target/generated-sources/org/apache/doris/
 cp -r fe/fe-core/target/generated-sources/cup/* fe/fe-core/target/generated-sources/
 rm -rf fe/fe-core/target/generated-sources/annotations/org/apache/doris/nereids
 rm -rf fe/fe-core/target/generated-sources/cup/
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


github-actions[bot] avatar Jun 23 '24 08:06 github-actions[bot]

run buildall

englefly avatar Jun 23 '24 08:06 englefly

TPC-H: Total hot run time: 39836 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc118e6b32d65d942a82eae2a6f69eefe29ab86d, data reload: false

------ Round 1 ----------------------------------
q1	17749	5380	4397	4397
q2	2375	192	204	192
q3	10461	1095	1085	1085
q4	10193	832	754	754
q5	7524	2656	2637	2637
q6	221	134	131	131
q7	953	610	601	601
q8	9225	2069	2054	2054
q9	8923	6453	6450	6450
q10	8951	3700	3708	3700
q11	441	244	229	229
q12	404	235	247	235
q13	17984	2975	2977	2975
q14	271	222	224	222
q15	523	479	474	474
q16	516	375	374	374
q17	959	722	696	696
q18	8149	7439	7410	7410
q19	4299	1468	1410	1410
q20	674	307	311	307
q21	4924	3169	3948	3169
q22	398	343	334	334
Total cold run time: 116117 ms
Total hot run time: 39836 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4350	4180	4204	4180
q2	365	269	279	269
q3	2941	2764	2773	2764
q4	1890	1643	1561	1561
q5	5259	5285	5278	5278
q6	214	126	126	126
q7	2121	1734	1737	1734
q8	3190	3329	3329	3329
q9	8339	8313	8370	8313
q10	3901	3663	3665	3663
q11	590	489	486	486
q12	754	630	605	605
q13	17464	2984	2974	2974
q14	281	261	241	241
q15	505	472	471	471
q16	468	409	412	409
q17	1784	1481	1479	1479
q18	7688	7414	7287	7287
q19	1673	1523	1506	1506
q20	1960	1772	1777	1772
q21	4921	4776	4690	4690
q22	603	552	550	550
Total cold run time: 71261 ms
Total hot run time: 53687 ms

doris-robot avatar Jun 23 '24 09:06 doris-robot

TPC-DS: Total hot run time: 171977 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc118e6b32d65d942a82eae2a6f69eefe29ab86d, data reload: false

query1	929	382	378	378
query2	6464	2379	2347	2347
query3	6653	211	218	211
query4	18928	17240	17377	17240
query5	4214	509	477	477
query6	260	166	172	166
query7	4610	303	302	302
query8	337	293	313	293
query9	8591	2405	2372	2372
query10	617	312	293	293
query11	10734	10135	9931	9931
query12	149	95	89	89
query13	1657	377	373	373
query14	10277	6124	7186	6124
query15	271	190	187	187
query16	8046	265	268	265
query17	1897	544	544	544
query18	2049	278	272	272
query19	202	156	156	156
query20	93	85	85	85
query21	206	127	132	127
query22	4286	3981	4054	3981
query23	33756	33030	33079	33030
query24	11683	2782	2832	2782
query25	631	367	360	360
query26	1817	155	153	153
query27	3001	323	324	323
query28	7627	2062	2052	2052
query29	1030	627	615	615
query30	275	156	155	155
query31	969	716	745	716
query32	97	56	59	56
query33	764	288	280	280
query34	999	482	466	466
query35	752	612	636	612
query36	1092	918	928	918
query37	187	70	70	70
query38	2888	2739	2720	2720
query39	880	799	801	799
query40	275	130	127	127
query41	55	52	52	52
query42	125	101	105	101
query43	606	575	526	526
query44	1267	739	737	737
query45	199	181	166	166
query46	1088	723	709	709
query47	1858	1758	1759	1758
query48	375	303	307	303
query49	1189	418	404	404
query50	783	391	407	391
query51	6853	6746	6761	6746
query52	107	98	93	93
query53	353	291	294	291
query54	927	439	446	439
query55	76	73	74	73
query56	320	272	283	272
query57	1119	1065	1049	1049
query58	258	244	250	244
query59	3462	3173	3108	3108
query60	309	292	288	288
query61	98	94	96	94
query62	642	451	472	451
query63	321	294	292	292
query64	9854	2240	1749	1749
query65	3167	3100	3126	3100
query66	1379	345	336	336
query67	15360	15167	15023	15023
query68	4643	543	547	543
query69	522	460	382	382
query70	1169	1134	1145	1134
query71	397	279	278	278
query72	7033	5250	5409	5250
query73	760	331	339	331
query74	5956	5570	5451	5451
query75	3423	2648	2697	2648
query76	2914	955	918	918
query77	448	304	302	302
query78	10325	9900	9757	9757
query79	2618	523	520	520
query80	2536	467	467	467
query81	570	228	219	219
query82	1034	105	105	105
query83	321	174	177	174
query84	265	89	91	89
query85	1305	281	271	271
query86	445	307	310	307
query87	3296	3071	3070	3070
query88	4000	2471	2476	2471
query89	484	375	386	375
query90	1853	192	193	192
query91	132	99	99	99
query92	139	53	50	50
query93	1953	513	512	512
query94	1352	181	182	181
query95	400	308	319	308
query96	606	270	275	270
query97	3266	3037	3030	3030
query98	238	202	190	190
query99	1218	849	841	841
Total cold run time: 277964 ms
Total hot run time: 171977 ms

doris-robot avatar Jun 23 '24 09:06 doris-robot

ClickBench: Total hot run time: 30.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc118e6b32d65d942a82eae2a6f69eefe29ab86d, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.49	0.50	0.49
query6	1.12	0.74	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.56	0.50	0.48
query10	0.55	0.54	0.57
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.60	0.58	0.60
query14	0.79	0.76	0.80
query15	0.83	0.81	0.81
query16	0.37	0.36	0.37
query17	1.02	1.00	0.95
query18	0.22	0.25	0.21
query19	1.86	1.73	1.72
query20	0.01	0.02	0.01
query21	15.42	0.64	0.64
query22	4.37	6.72	1.93
query23	18.29	1.40	1.23
query24	2.16	0.23	0.23
query25	0.16	0.08	0.09
query26	0.28	0.17	0.17
query27	0.07	0.08	0.07
query28	13.24	1.01	0.99
query29	12.63	3.42	3.40
query30	0.27	0.06	0.07
query31	2.84	0.38	0.37
query32	3.30	0.47	0.47
query33	2.89	2.91	2.85
query34	17.23	4.44	4.41
query35	4.49	4.64	4.51
query36	0.64	0.48	0.50
query37	0.20	0.16	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.04
query40	0.19	0.15	0.14
query41	0.11	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.89 s
Total hot run time: 30.62 s

doris-robot avatar Jun 23 '24 09:06 doris-robot

run feut

englefly avatar Jun 24 '24 02:06 englefly

run buildall

englefly avatar Jun 24 '24 12:06 englefly

run buildall

englefly avatar Jun 24 '24 12:06 englefly

run buildall

englefly avatar Jun 25 '24 01:06 englefly

TPC-H: Total hot run time: 39685 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ab94f61e7cdacd48ac9964ab61ef4bcdca309de, data reload: false

------ Round 1 ----------------------------------
q1	17634	4560	4285	4285
q2	2018	193	191	191
q3	10546	1187	1022	1022
q4	10253	835	766	766
q5	7494	2646	2646	2646
q6	242	135	134	134
q7	962	594	613	594
q8	9228	2063	2062	2062
q9	9049	6440	6472	6440
q10	8877	3766	3742	3742
q11	446	247	237	237
q12	390	235	217	217
q13	18551	2987	2962	2962
q14	268	218	220	218
q15	516	470	493	470
q16	534	374	383	374
q17	973	639	706	639
q18	8143	7506	7451	7451
q19	8000	1485	1456	1456
q20	677	319	337	319
q21	4895	3123	3886	3123
q22	397	337	339	337
Total cold run time: 120093 ms
Total hot run time: 39685 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4347	4255	4248	4248
q2	370	269	268	268
q3	3110	2895	2929	2895
q4	1985	1775	1689	1689
q5	5505	5509	5489	5489
q6	223	136	141	136
q7	2262	1832	1932	1832
q8	3301	3473	3379	3379
q9	8729	8845	8718	8718
q10	4195	3693	3849	3693
q11	622	506	500	500
q12	827	643	641	641
q13	16092	3189	3188	3188
q14	316	282	262	262
q15	528	489	496	489
q16	492	443	430	430
q17	1789	1504	1491	1491
q18	8221	8071	8029	8029
q19	1791	1669	1700	1669
q20	2137	1851	1848	1848
q21	5009	4878	4829	4829
q22	638	572	568	568
Total cold run time: 72489 ms
Total hot run time: 56291 ms

doris-robot avatar Jun 25 '24 01:06 doris-robot

TPC-DS: Total hot run time: 174479 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ab94f61e7cdacd48ac9964ab61ef4bcdca309de, data reload: false

query1	905	381	387	381
query2	6454	2374	2465	2374
query3	6626	201	195	195
query4	19064	17388	17269	17269
query5	3694	462	468	462
query6	235	169	175	169
query7	4579	303	296	296
query8	311	299	315	299
query9	8599	2464	2450	2450
query10	572	301	267	267
query11	10451	9909	9968	9909
query12	114	81	86	81
query13	1631	369	384	369
query14	9583	7695	7871	7695
query15	244	199	187	187
query16	7953	267	262	262
query17	1919	550	546	546
query18	2081	282	279	279
query19	203	154	162	154
query20	90	87	83	83
query21	212	130	125	125
query22	4376	4071	4116	4071
query23	33873	33756	33544	33544
query24	11910	3041	2889	2889
query25	660	410	412	410
query26	1744	162	158	158
query27	3174	346	338	338
query28	7604	2170	2157	2157
query29	1075	628	645	628
query30	242	164	156	156
query31	955	758	752	752
query32	109	55	56	55
query33	789	316	311	311
query34	1065	488	480	480
query35	779	683	650	650
query36	1140	974	988	974
query37	301	79	76	76
query38	3037	2847	2830	2830
query39	931	868	814	814
query40	264	129	129	129
query41	58	55	52	52
query42	123	97	99	97
query43	580	596	561	561
query44	1205	738	716	716
query45	200	167	171	167
query46	1082	746	766	746
query47	1875	1743	1793	1743
query48	388	312	303	303
query49	958	430	428	428
query50	784	392	402	392
query51	6973	6803	6817	6803
query52	118	91	100	91
query53	375	286	292	286
query54	949	453	455	453
query55	79	74	75	74
query56	306	273	273	273
query57	1150	1090	1053	1053
query58	254	243	251	243
query59	3570	3243	3000	3000
query60	309	286	315	286
query61	112	112	110	110
query62	702	437	453	437
query63	330	293	296	293
query64	9626	2251	1751	1751
query65	3199	3098	3126	3098
query66	1232	334	329	329
query67	15619	14865	14876	14865
query68	8401	549	544	544
query69	689	423	401	401
query70	1149	1153	1119	1119
query71	521	276	277	276
query72	9229	5522	5166	5166
query73	1255	329	325	325
query74	6158	5513	5496	5496
query75	5065	2660	2635	2635
query76	5007	915	884	884
query77	806	322	304	304
query78	10586	9914	9730	9730
query79	8289	538	508	508
query80	1007	485	478	478
query81	550	217	218	217
query82	756	118	101	101
query83	345	173	165	165
query84	272	87	90	87
query85	1374	279	272	272
query86	405	334	304	304
query87	3310	3078	3087	3078
query88	4968	2452	2445	2445
query89	546	390	373	373
query90	1997	193	186	186
query91	127	100	104	100
query92	62	51	50	50
query93	5698	505	499	499
query94	1183	192	188	188
query95	406	309	316	309
query96	618	273	267	267
query97	3292	3041	3043	3041
query98	222	200	190	190
query99	1203	817	838	817
Total cold run time: 297200 ms
Total hot run time: 174479 ms

doris-robot avatar Jun 25 '24 01:06 doris-robot

ClickBench: Total hot run time: 30.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ab94f61e7cdacd48ac9964ab61ef4bcdca309de, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.71	0.08	0.09
query5	0.51	0.49	0.50
query6	1.13	0.73	0.71
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.48	0.49
query10	0.53	0.54	0.54
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.75	0.79	0.79
query15	0.83	0.82	0.81
query16	0.37	0.37	0.35
query17	1.06	1.04	1.02
query18	0.24	0.23	0.25
query19	1.88	1.78	1.75
query20	0.02	0.01	0.01
query21	15.39	0.67	0.66
query22	4.11	6.86	2.03
query23	18.32	1.45	1.34
query24	2.07	0.24	0.22
query25	0.15	0.08	0.09
query26	0.28	0.18	0.17
query27	0.08	0.08	0.07
query28	13.23	1.02	1.02
query29	12.68	3.24	3.22
query30	0.26	0.07	0.06
query31	2.87	0.40	0.39
query32	3.26	0.46	0.47
query33	2.88	2.86	2.95
query34	17.12	4.39	4.34
query35	4.44	4.49	4.46
query36	0.65	0.46	0.46
query37	0.18	0.16	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.03
query40	0.18	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.42 s
Total hot run time: 30.67 s

doris-robot avatar Jun 25 '24 01:06 doris-robot

run buildall

englefly avatar Jun 25 '24 09:06 englefly

TPC-H: Total hot run time: 39754 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9bd7919c7176908dc6ac0306282e833a26a7dc9, data reload: false

------ Round 1 ----------------------------------
q1	17601	4283	4229	4229
q2	2011	184	185	184
q3	10483	1114	1138	1114
q4	10183	863	841	841
q5	7461	2656	2657	2656
q6	224	134	130	130
q7	960	588	596	588
q8	9213	2039	2040	2039
q9	9079	6503	6414	6414
q10	8910	3689	3701	3689
q11	475	240	243	240
q12	408	223	221	221
q13	17795	2971	2991	2971
q14	273	217	225	217
q15	525	487	487	487
q16	478	375	379	375
q17	968	715	682	682
q18	7842	7394	7402	7394
q19	5486	1517	1472	1472
q20	639	304	299	299
q21	4802	3176	3332	3176
q22	391	336	338	336
Total cold run time: 116207 ms
Total hot run time: 39754 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4441	4249	4272	4249
q2	368	279	265	265
q3	2964	2719	2926	2719
q4	2024	1708	1703	1703
q5	5582	5471	5519	5471
q6	222	136	134	134
q7	2235	1814	1879	1814
q8	3283	3430	3390	3390
q9	8655	8637	8850	8637
q10	4081	3935	3695	3695
q11	597	523	511	511
q12	814	634	630	630
q13	16154	3178	3187	3178
q14	299	270	279	270
q15	536	507	488	488
q16	472	420	423	420
q17	1809	1528	1499	1499
q18	8128	7989	7842	7842
q19	1843	1589	1554	1554
q20	3034	1896	1884	1884
q21	5165	5072	4792	4792
q22	615	588	561	561
Total cold run time: 73321 ms
Total hot run time: 55706 ms

doris-robot avatar Jun 25 '24 10:06 doris-robot

TPC-DS: Total hot run time: 174621 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9bd7919c7176908dc6ac0306282e833a26a7dc9, data reload: false

query1	912	389	384	384
query2	6465	2401	2433	2401
query3	6622	198	213	198
query4	19823	17449	17129	17129
query5	3624	462	471	462
query6	235	162	156	156
query7	4589	296	291	291
query8	322	303	298	298
query9	8660	2391	2374	2374
query10	570	298	271	271
query11	10481	10104	9962	9962
query12	112	79	82	79
query13	1631	381	385	381
query14	9427	7981	7633	7633
query15	252	190	191	190
query16	7985	278	271	271
query17	1895	576	535	535
query18	2053	283	292	283
query19	204	157	160	157
query20	94	83	88	83
query21	209	131	126	126
query22	4353	4202	3855	3855
query23	33964	33689	33787	33689
query24	11063	2916	2910	2910
query25	637	409	407	407
query26	871	170	163	163
query27	2324	334	325	325
query28	6347	2154	2152	2152
query29	905	669	672	669
query30	253	159	160	159
query31	953	764	773	764
query32	96	54	56	54
query33	793	304	301	301
query34	1030	470	478	470
query35	797	656	637	637
query36	1135	991	986	986
query37	152	77	84	77
query38	2941	2879	2878	2878
query39	902	845	845	845
query40	218	136	131	131
query41	58	54	54	54
query42	112	104	99	99
query43	630	546	538	538
query44	1221	735	735	735
query45	199	165	164	164
query46	1069	721	709	709
query47	1833	1828	1777	1777
query48	387	303	308	303
query49	858	424	437	424
query50	783	395	387	387
query51	6922	6797	6885	6797
query52	110	92	93	92
query53	358	289	287	287
query54	897	476	453	453
query55	76	74	76	74
query56	298	284	292	284
query57	1168	1072	1046	1046
query58	339	250	246	246
query59	3638	3288	3598	3288
query60	291	270	275	270
query61	108	95	99	95
query62	597	436	469	436
query63	329	291	289	289
query64	8683	2263	1743	1743
query65	3301	3127	3123	3123
query66	776	324	322	322
query67	15639	14931	14977	14931
query68	8182	576	552	552
query69	731	473	392	392
query70	1213	1155	1045	1045
query71	513	283	276	276
query72	8761	5454	5315	5315
query73	1134	333	324	324
query74	5837	5502	5438	5438
query75	5185	2695	2705	2695
query76	4736	997	996	996
query77	789	317	298	298
query78	10475	9829	9680	9680
query79	8455	511	516	511
query80	1015	462	475	462
query81	547	219	217	217
query82	753	109	98	98
query83	359	168	164	164
query84	269	84	85	84
query85	1293	290	269	269
query86	399	325	296	296
query87	3297	3075	3086	3075
query88	4963	2388	2345	2345
query89	532	377	396	377
query90	1941	186	185	185
query91	129	99	100	99
query92	62	51	54	51
query93	6237	518	518	518
query94	1169	189	188	188
query95	407	317	320	317
query96	599	275	265	265
query97	3221	2964	3027	2964
query98	215	199	207	199
query99	1203	855	824	824
Total cold run time: 291200 ms
Total hot run time: 174621 ms

doris-robot avatar Jun 25 '24 10:06 doris-robot

ClickBench: Total hot run time: 31.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9bd7919c7176908dc6ac0306282e833a26a7dc9, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.10	0.10
query5	0.50	0.57	0.49
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.54	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.78	0.79	0.80
query15	0.83	0.81	0.81
query16	0.36	0.36	0.37
query17	1.03	0.94	0.98
query18	0.22	0.25	0.22
query19	1.78	1.70	1.68
query20	0.01	0.01	0.01
query21	15.43	0.65	0.65
query22	4.30	6.03	2.55
query23	18.30	1.29	1.28
query24	2.07	0.25	0.23
query25	0.16	0.09	0.09
query26	0.26	0.18	0.17
query27	0.09	0.08	0.07
query28	13.20	1.00	1.01
query29	12.60	3.28	3.30
query30	0.25	0.07	0.05
query31	2.87	0.38	0.39
query32	3.27	0.47	0.48
query33	2.81	2.96	2.94
query34	17.05	4.43	4.45
query35	4.54	4.44	4.50
query36	0.67	0.47	0.46
query37	0.18	0.15	0.16
query38	0.14	0.14	0.14
query39	0.04	0.03	0.04
query40	0.18	0.16	0.14
query41	0.09	0.05	0.05
query42	0.06	0.06	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.32 s
Total hot run time: 31.23 s

doris-robot avatar Jun 25 '24 10:06 doris-robot

run buildall

englefly avatar Jun 25 '24 15:06 englefly

run buildall

englefly avatar Jun 26 '24 06:06 englefly

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Jun 26 '24 06:06 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Jun 26 '24 06:06 github-actions[bot]

TPC-H: Total hot run time: 39902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3680b286c0195d5da2d12d9924228777bfe4143c, data reload: false

------ Round 1 ----------------------------------
q1	17615	4449	4239	4239
q2	2022	182	185	182
q3	10451	1157	1160	1157
q4	10191	808	736	736
q5	7484	2668	2659	2659
q6	219	131	131	131
q7	946	593	595	593
q8	9215	2089	2093	2089
q9	8926	6516	6471	6471
q10	9043	3717	3736	3717
q11	456	245	237	237
q12	461	229	227	227
q13	17883	3010	2988	2988
q14	274	212	227	212
q15	515	488	484	484
q16	505	384	378	378
q17	963	655	730	655
q18	8119	7491	7481	7481
q19	5179	1405	1470	1405
q20	659	324	323	323
q21	5346	3201	3887	3201
q22	392	337	342	337
Total cold run time: 116864 ms
Total hot run time: 39902 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4484	4235	4256	4235
q2	381	261	262	261
q3	2989	2934	2846	2846
q4	2011	1695	1711	1695
q5	5576	5510	5527	5510
q6	217	129	130	129
q7	2212	1906	1896	1896
q8	3231	3418	3444	3418
q9	8729	8694	8803	8694
q10	4124	3866	3716	3716
q11	591	520	513	513
q12	833	664	663	663
q13	15811	3202	3170	3170
q14	306	286	283	283
q15	518	496	475	475
q16	496	439	440	439
q17	1845	1524	1510	1510
q18	8203	8138	7795	7795
q19	4415	1702	1684	1684
q20	2093	1849	1863	1849
q21	5072	4881	4948	4881
q22	619	514	545	514
Total cold run time: 74756 ms
Total hot run time: 56176 ms

doris-robot avatar Jun 26 '24 07:06 doris-robot

TPC-DS: Total hot run time: 173925 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3680b286c0195d5da2d12d9924228777bfe4143c, data reload: false

query1	910	382	383	382
query2	6422	2437	2189	2189
query3	6623	210	224	210
query4	19720	17534	17495	17495
query5	3609	465	469	465
query6	238	170	160	160
query7	4575	305	297	297
query8	324	281	274	274
query9	8716	2446	2450	2446
query10	568	316	299	299
query11	10509	9936	10104	9936
query12	114	85	85	85
query13	1633	364	360	360
query14	10156	7074	7288	7074
query15	234	182	184	182
query16	7696	275	261	261
query17	1860	516	520	516
query18	1867	275	269	269
query19	194	153	166	153
query20	93	86	83	83
query21	222	130	139	130
query22	4458	4247	3940	3940
query23	33878	33585	33727	33585
query24	10948	2878	2882	2878
query25	646	374	381	374
query26	1148	158	152	152
query27	2473	319	333	319
query28	7215	2153	2167	2153
query29	926	625	604	604
query30	249	154	158	154
query31	965	747	777	747
query32	85	56	52	52
query33	761	290	296	290
query34	1017	470	481	470
query35	763	644	641	641
query36	1105	995	976	976
query37	172	79	74	74
query38	3004	2901	2820	2820
query39	926	836	828	828
query40	229	126	129	126
query41	52	53	72	53
query42	115	106	101	101
query43	576	542	525	525
query44	1204	725	733	725
query45	190	167	159	159
query46	1088	726	736	726
query47	1876	1779	1793	1779
query48	373	298	311	298
query49	844	409	414	409
query50	778	379	386	379
query51	6899	6885	6731	6731
query52	105	102	93	93
query53	356	290	284	284
query54	867	444	452	444
query55	85	73	74	73
query56	289	266	282	266
query57	1093	1014	1056	1014
query58	250	248	251	248
query59	3366	3022	2997	2997
query60	287	280	273	273
query61	94	104	109	104
query62	608	447	451	447
query63	322	296	292	292
query64	8882	2277	1740	1740
query65	3157	3098	3089	3089
query66	747	336	325	325
query67	15438	14944	14792	14792
query68	4487	534	533	533
query69	460	301	321	301
query70	1186	1108	1141	1108
query71	412	275	289	275
query72	7064	5610	5711	5610
query73	753	328	323	323
query74	5968	5628	5556	5556
query75	3363	2705	2681	2681
query76	2629	944	931	931
query77	423	303	299	299
query78	10417	9824	9722	9722
query79	2108	507	509	507
query80	1695	467	476	467
query81	570	223	220	220
query82	994	101	100	100
query83	275	169	169	169
query84	242	84	85	84
query85	1148	285	278	278
query86	395	337	311	311
query87	3265	3108	3119	3108
query88	3526	2485	2454	2454
query89	476	390	388	388
query90	1691	192	192	192
query91	128	100	103	100
query92	57	48	48	48
query93	1486	499	501	499
query94	1032	195	189	189
query95	398	319	320	319
query96	588	276	272	272
query97	3183	3028	3062	3028
query98	220	197	199	197
query99	1171	841	865	841
Total cold run time: 269002 ms
Total hot run time: 173925 ms

doris-robot avatar Jun 26 '24 07:06 doris-robot

ClickBench: Total hot run time: 30.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3680b286c0195d5da2d12d9924228777bfe4143c, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.22	0.04	0.05
query4	1.68	0.07	0.08
query5	0.51	0.47	0.50
query6	1.14	0.74	0.72
query7	0.01	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.54	0.55	0.55
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.76	0.78	0.79
query15	0.84	0.82	0.82
query16	0.36	0.37	0.37
query17	0.98	0.98	0.98
query18	0.22	0.25	0.21
query19	1.93	1.83	1.79
query20	0.01	0.01	0.01
query21	15.42	0.64	0.65
query22	4.54	7.63	1.43
query23	18.29	1.35	1.22
query24	2.13	0.22	0.23
query25	0.16	0.09	0.09
query26	0.27	0.17	0.18
query27	0.08	0.09	0.08
query28	13.24	1.02	0.99
query29	12.61	3.30	3.29
query30	0.25	0.06	0.05
query31	2.86	0.38	0.38
query32	3.29	0.47	0.46
query33	2.90	2.92	2.96
query34	17.27	4.44	4.44
query35	4.48	4.49	4.46
query36	0.66	0.47	0.46
query37	0.19	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.18	0.15	0.14
query41	0.09	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.98 s
Total hot run time: 30.13 s

doris-robot avatar Jun 26 '24 07:06 doris-robot

run p0

englefly avatar Jun 26 '24 10:06 englefly

run buildall

englefly avatar Jun 27 '24 03:06 englefly