doris icon indicating copy to clipboard operation
doris copied to clipboard

[feat](nereids) Add constant propagation

Open yujun777 opened this issue 7 months ago • 170 comments

What problem does this PR solve?

support constant propagation, for example: a = 10 and a + b > 30 will rewrite to a = 10 and 10 + b > 30,

when processing a plan, it will collect all its children's equal sets and constants uniforms, then use them and the plan's expressions to infer more equal sets and constants uniforms, finally use the combine uniforms to replace this plan's expression's slot with literals.

TODO:

  1. process with LogicalRepeat;
  2. adjust the rule order;
  3. add more test, include agg, join, subquery, and logical repeat;

Check List (For Author)

  • Test

    • [x] Regression test
    • [x] Unit Test
    • [ ] Manual test (add detailed scripts or steps below)
    • [ ] No need to test or manual test. Explain why:
      • [ ] This is a refactor/code format and no logic has been changed.
      • [ ] Previous test can cover this change.
      • [ ] No code files have been changed.
      • [ ] Other reason
  • Behavior changed:

    • [ ] No.
    • [ ] Yes.
  • Does this need documentation?

    • [ ] No.
    • [ ] Yes.

Check List (For Reviewer who merge this PR)

  • [ ] Confirm the release note
  • [ ] Confirm test cases
  • [ ] Confirm document
  • [ ] Add branch pick label

yujun777 avatar Apr 11 '25 03:04 yujun777

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Thearas avatar Apr 11 '25 03:04 Thearas

run buildall

yujun777 avatar Apr 11 '25 03:04 yujun777

run buildall

yujun777 avatar Apr 11 '25 03:04 yujun777

run buildall

yujun777 avatar Apr 11 '25 04:04 yujun777

TPC-H: Total hot run time: 33911 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f28a750f710b6d3e443dd61dcdbbcede2b504357, data reload: false

------ Round 1 ----------------------------------
q1	26129	5091	4936	4936
q2	2062	285	183	183
q3	10393	1201	686	686
q4	10250	997	528	528
q5	7531	2359	2311	2311
q6	181	166	132	132
q7	910	734	626	626
q8	9337	1256	1075	1075
q9	6824	5111	5037	5037
q10	6797	2287	1908	1908
q11	471	289	279	279
q12	350	351	225	225
q13	17785	3679	3097	3097
q14	224	228	213	213
q15	554	498	498	498
q16	619	619	579	579
q17	593	854	379	379
q18	7542	7025	7170	7025
q19	1257	938	536	536
q20	330	326	218	218
q21	3977	3383	2465	2465
q22	1047	1034	975	975
Total cold run time: 115163 ms
Total hot run time: 33911 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5111	5066	5042	5042
q2	233	318	225	225
q3	2144	2615	2254	2254
q4	1386	1777	1390	1390
q5	4486	4397	4375	4375
q6	214	171	132	132
q7	1957	1908	1741	1741
q8	2593	2665	2555	2555
q9	7299	7241	6990	6990
q10	2995	3159	2746	2746
q11	572	501	518	501
q12	679	759	635	635
q13	3545	3906	3336	3336
q14	275	288	266	266
q15	527	499	464	464
q16	637	676	660	660
q17	1140	1555	1374	1374
q18	7703	7689	7401	7401
q19	801	820	944	820
q20	1985	2022	1851	1851
q21	5117	4805	5013	4805
q22	1114	1049	1056	1049
Total cold run time: 52513 ms
Total hot run time: 50612 ms

doris-robot avatar Apr 11 '25 04:04 doris-robot

TPC-DS: Total hot run time: 191908 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f28a750f710b6d3e443dd61dcdbbcede2b504357, data reload: false

query1	1403	1070	1036	1036
query2	6115	1950	1891	1891
query3	11007	4435	4376	4376
query4	53065	25286	23337	23337
query5	5301	485	445	445
query6	369	206	211	206
query7	5004	480	275	275
query8	323	286	259	259
query9	6352	2621	2646	2621
query10	447	324	264	264
query11	15015	14969	14779	14779
query12	150	107	109	107
query13	1094	500	370	370
query14	10194	6597	6376	6376
query15	206	188	190	188
query16	7133	682	504	504
query17	1117	753	602	602
query18	1535	398	335	335
query19	206	202	165	165
query20	135	123	119	119
query21	223	128	108	108
query22	4327	4398	4306	4306
query23	34190	33474	33648	33474
query24	6566	2444	2424	2424
query25	488	459	404	404
query26	714	265	152	152
query27	2251	502	335	335
query28	2965	2437	2433	2433
query29	579	563	436	436
query30	282	225	190	190
query31	827	818	758	758
query32	74	66	67	66
query33	473	392	322	322
query34	833	859	517	517
query35	835	877	761	761
query36	960	989	909	909
query37	121	105	80	80
query38	4319	4269	4179	4179
query39	1474	1424	1435	1424
query40	211	131	108	108
query41	56	55	57	55
query42	130	117	117	117
query43	524	513	498	498
query44	1382	838	846	838
query45	183	186	179	179
query46	828	1026	654	654
query47	1828	1863	1824	1824
query48	386	424	314	314
query49	692	501	451	451
query50	655	727	398	398
query51	4233	4255	4372	4255
query52	113	113	98	98
query53	243	253	182	182
query54	579	599	517	517
query55	84	84	83	83
query56	316	303	292	292
query57	1161	1196	1162	1162
query58	276	291	261	261
query59	2730	3005	2746	2746
query60	352	326	338	326
query61	136	133	124	124
query62	743	716	692	692
query63	233	185	188	185
query64	1706	1016	669	669
query65	4356	4259	4248	4248
query66	819	395	322	322
query67	15661	15490	15219	15219
query68	7918	872	511	511
query69	586	306	269	269
query70	1213	1143	1097	1097
query71	497	314	307	307
query72	5456	4779	4952	4779
query73	1306	694	360	360
query74	9143	9198	8690	8690
query75	3675	3177	2685	2685
query76	4292	1081	649	649
query77	623	368	288	288
query78	8995	8883	8327	8327
query79	2059	877	564	564
query80	597	519	458	458
query81	485	250	219	219
query82	465	130	97	97
query83	268	258	238	238
query84	295	112	84	84
query85	870	324	274	274
query86	427	301	303	301
query87	4485	4591	4406	4406
query88	3664	2255	2246	2246
query89	403	316	287	287
query90	1839	217	218	217
query91	144	147	116	116
query92	76	64	59	59
query93	1757	995	572	572
query94	677	397	314	314
query95	379	299	284	284
query96	494	567	283	283
query97	3172	3206	3098	3098
query98	265	200	203	200
query99	1416	1402	1279	1279
Total cold run time: 297229 ms
Total hot run time: 191908 ms

doris-robot avatar Apr 11 '25 05:04 doris-robot

ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f28a750f710b6d3e443dd61dcdbbcede2b504357, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.10	0.10
query3	0.26	0.19	0.19
query4	1.59	0.19	0.11
query5	0.56	0.54	0.54
query6	1.20	0.72	0.71
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.58	0.52	0.52
query10	0.59	0.61	0.59
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.60	0.60
query14	2.67	2.79	2.80
query15	0.93	0.86	0.84
query16	0.40	0.37	0.38
query17	1.05	1.03	1.02
query18	0.22	0.19	0.19
query19	2.05	1.87	1.83
query20	0.01	0.01	0.02
query21	15.36	0.88	0.56
query22	0.75	1.43	0.73
query23	14.76	1.45	0.66
query24	6.81	1.32	0.51
query25	0.50	0.24	0.18
query26	0.71	0.17	0.14
query27	0.05	0.05	0.05
query28	8.61	0.96	0.44
query29	12.53	3.98	3.31
query30	0.25	0.10	0.07
query31	2.81	0.58	0.40
query32	3.23	0.55	0.47
query33	3.00	3.02	3.03
query34	15.70	5.05	4.45
query35	4.55	4.49	4.51
query36	0.65	0.51	0.48
query37	0.09	0.07	0.07
query38	0.05	0.04	0.05
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 104 s
Total hot run time: 31 s

doris-robot avatar Apr 11 '25 05:04 doris-robot

run p0

yujun777 avatar Apr 11 '25 07:04 yujun777

run cloud_p0

yujun777 avatar Apr 11 '25 07:04 yujun777

run p0

yujun777 avatar Apr 11 '25 11:04 yujun777

run cloud_p0

yujun777 avatar Apr 11 '25 11:04 yujun777

run buildall

yujun777 avatar Apr 12 '25 03:04 yujun777

TPC-H: Total hot run time: 34121 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 679002894d5fb311f725de08d1ffda48e95234fc, data reload: false

------ Round 1 ----------------------------------
q1	25911	5023	5016	5016
q2	2071	268	180	180
q3	10398	1243	689	689
q4	10228	1011	540	540
q5	7539	2370	2324	2324
q6	186	167	132	132
q7	933	733	631	631
q8	9304	1216	1085	1085
q9	6827	5093	5127	5093
q10	6876	2310	1873	1873
q11	494	280	278	278
q12	349	344	222	222
q13	17793	3646	3084	3084
q14	227	227	206	206
q15	542	507	495	495
q16	642	611	574	574
q17	577	853	357	357
q18	7541	7254	7089	7089
q19	1442	949	577	577
q20	339	328	227	227
q21	4027	3405	2480	2480
q22	1063	1024	969	969
Total cold run time: 115309 ms
Total hot run time: 34121 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5149	5121	5046	5046
q2	244	329	235	235
q3	2211	2705	2345	2345
q4	1463	1828	1449	1449
q5	4505	4357	4386	4357
q6	217	177	138	138
q7	2009	1915	1737	1737
q8	2565	2623	2525	2525
q9	7261	7270	7061	7061
q10	2967	3159	2766	2766
q11	572	519	496	496
q12	712	775	627	627
q13	3504	3894	3348	3348
q14	311	318	272	272
q15	520	481	494	481
q16	660	688	650	650
q17	1152	1492	1404	1404
q18	7718	7668	7576	7576
q19	816	778	837	778
q20	1927	1951	1804	1804
q21	5218	4967	4823	4823
q22	1094	1062	1023	1023
Total cold run time: 52795 ms
Total hot run time: 50941 ms

doris-robot avatar Apr 12 '25 04:04 doris-robot

TPC-DS: Total hot run time: 192177 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 679002894d5fb311f725de08d1ffda48e95234fc, data reload: false

query1	1444	1099	1040	1040
query2	6374	1879	1879	1879
query3	10971	4518	4340	4340
query4	53576	25477	23557	23557
query5	5040	517	491	491
query6	341	215	207	207
query7	4877	499	282	282
query8	333	262	260	260
query9	5471	2590	2592	2590
query10	420	334	262	262
query11	14901	14894	14833	14833
query12	158	112	110	110
query13	1040	536	384	384
query14	10120	6436	6314	6314
query15	204	201	186	186
query16	7060	680	465	465
query17	1062	750	617	617
query18	1554	412	312	312
query19	197	197	167	167
query20	133	127	130	127
query21	209	124	107	107
query22	4511	4415	4265	4265
query23	34247	33527	33408	33408
query24	6614	2420	2421	2420
query25	483	473	397	397
query26	680	279	163	163
query27	2288	525	335	335
query28	3115	2465	2431	2431
query29	570	597	457	457
query30	284	227	206	206
query31	817	857	719	719
query32	75	64	63	63
query33	473	375	330	330
query34	776	862	526	526
query35	811	838	751	751
query36	949	1012	887	887
query37	124	100	81	81
query38	4267	4272	4205	4205
query39	1472	1436	1444	1436
query40	214	131	114	114
query41	62	59	59	59
query42	126	113	111	111
query43	514	512	496	496
query44	1341	844	864	844
query45	188	176	169	169
query46	875	1034	646	646
query47	1877	1889	1792	1792
query48	371	413	310	310
query49	696	527	432	432
query50	662	699	410	410
query51	4267	4344	4166	4166
query52	109	103	106	103
query53	239	253	194	194
query54	583	596	540	540
query55	86	82	83	82
query56	311	329	300	300
query57	1199	1210	1143	1143
query58	274	272	268	268
query59	2727	2784	2735	2735
query60	340	333	322	322
query61	135	130	127	127
query62	768	734	663	663
query63	226	193	186	186
query64	1506	1062	685	685
query65	4422	4246	4229	4229
query66	704	393	297	297
query67	15959	15474	15289	15289
query68	4572	894	524	524
query69	509	311	273	273
query70	1168	1116	1071	1071
query71	449	311	296	296
query72	5717	4859	4870	4859
query73	781	605	344	344
query74	8967	9110	9018	9018
query75	3485	3239	2746	2746
query76	3439	1092	650	650
query77	561	466	284	284
query78	9059	9215	8339	8339
query79	1712	884	568	568
query80	662	521	449	449
query81	477	259	237	237
query82	231	129	95	95
query83	254	248	241	241
query84	296	104	85	85
query85	687	330	275	275
query86	319	310	289	289
query87	4406	4475	4313	4313
query88	2782	2222	2298	2222
query89	387	331	279	279
query90	1837	212	213	212
query91	154	140	112	112
query92	75	67	58	58
query93	2198	974	567	567
query94	652	423	304	304
query95	370	295	297	295
query96	490	582	276	276
query97	3155	3236	3119	3119
query98	236	205	205	205
query99	1301	1396	1250	1250
Total cold run time: 290210 ms
Total hot run time: 192177 ms

doris-robot avatar Apr 12 '25 04:04 doris-robot

ClickBench: Total hot run time: 30.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 679002894d5fb311f725de08d1ffda48e95234fc, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.12
query3	0.26	0.20	0.19
query4	1.59	0.20	0.19
query5	0.61	0.60	0.58
query6	1.18	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.51	0.51
query10	0.58	0.57	0.59
query11	0.15	0.10	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.61
query14	2.72	2.72	2.70
query15	0.94	0.86	0.85
query16	0.40	0.40	0.38
query17	1.05	1.04	1.00
query18	0.21	0.19	0.20
query19	1.84	1.92	1.86
query20	0.02	0.01	0.02
query21	15.36	0.88	0.55
query22	0.76	1.08	0.60
query23	15.12	1.43	0.64
query24	7.19	1.28	0.58
query25	0.46	0.20	0.07
query26	0.61	0.17	0.15
query27	0.05	0.05	0.05
query28	8.66	0.90	0.45
query29	12.58	3.98	3.30
query30	0.25	0.09	0.06
query31	2.81	0.58	0.39
query32	3.24	0.56	0.47
query33	3.05	3.20	3.14
query34	15.78	5.08	4.47
query35	4.53	4.55	4.54
query36	0.65	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 104.71 s
Total hot run time: 30.98 s

doris-robot avatar Apr 12 '25 04:04 doris-robot

run buildall

yujun777 avatar Apr 15 '25 09:04 yujun777

TPC-H: Total hot run time: 33924 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2930d988d2c992f3fe1a6254e06d831d20a0a171, data reload: false

------ Round 1 ----------------------------------
q1	25910	5004	4991	4991
q2	2083	290	185	185
q3	10377	1206	684	684
q4	10222	1019	564	564
q5	7539	2319	2323	2319
q6	186	162	134	134
q7	912	714	607	607
q8	9300	1236	1051	1051
q9	6777	5082	5084	5082
q10	6801	2298	1896	1896
q11	485	290	267	267
q12	349	345	212	212
q13	17778	3698	3105	3105
q14	237	228	213	213
q15	527	478	493	478
q16	625	611	572	572
q17	572	843	351	351
q18	7435	7079	7135	7079
q19	1251	947	549	549
q20	336	322	208	208
q21	3888	3366	2416	2416
q22	1071	1040	961	961
Total cold run time: 114661 ms
Total hot run time: 33924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5068	5379	5115	5115
q2	237	326	232	232
q3	2271	2702	2285	2285
q4	1498	1844	1457	1457
q5	4501	4465	4443	4443
q6	214	170	130	130
q7	2075	1979	1814	1814
q8	2647	2569	2588	2569
q9	7430	7389	7256	7256
q10	3001	3169	2739	2739
q11	576	522	494	494
q12	701	762	628	628
q13	3463	3876	3335	3335
q14	301	307	296	296
q15	522	497	477	477
q16	656	702	646	646
q17	1147	1513	1409	1409
q18	7759	7512	7414	7414
q19	795	775	779	775
q20	1925	1935	1810	1810
q21	5321	4930	4772	4772
q22	1125	1049	1059	1049
Total cold run time: 53233 ms
Total hot run time: 51145 ms

doris-robot avatar Apr 15 '25 09:04 doris-robot

TPC-DS: Total hot run time: 191330 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2930d988d2c992f3fe1a6254e06d831d20a0a171, data reload: false

============================================
query1	1396	1103	1076	1076
query2	6176	1879	1908	1879
query3	11135	4711	4564	4564
query4	25518	23685	22950	22950
query5	3674	638	500	500
query6	301	199	199	199
query7	3987	492	277	277
query8	295	241	231	231
query9	8464	2572	2616	2572
query10	471	322	281	281
query11	15866	15370	14807	14807
query12	183	114	112	112
query13	1564	528	417	417
query14	9215	6072	6122	6072
query15	206	191	170	170
query16	7283	649	474	474
query17	1200	739	598	598
query18	2002	429	322	322
query19	200	201	167	167
query20	130	127	126	126
query21	207	131	108	108
query22	4480	4516	4476	4476
query23	34278	33740	33460	33460
query24	8642	2421	2404	2404
query25	515	485	398	398
query26	718	266	161	161
query27	2904	498	327	327
query28	4657	2457	2424	2424
query29	649	568	443	443
query30	278	219	193	193
query31	868	834	755	755
query32	76	84	70	70
query33	537	352	328	328
query34	803	883	510	510
query35	822	841	777	777
query36	975	1000	895	895
query37	133	105	80	80
query38	4226	4246	4246	4246
query39	1497	1472	1460	1460
query40	212	119	106	106
query41	62	64	53	53
query42	125	118	112	112
query43	502	495	481	481
query44	1305	813	844	813
query45	188	175	175	175
query46	851	1024	651	651
query47	1845	1859	1760	1760
query48	383	413	311	311
query49	query50	653	691	413	413
query51	4208	4264	4230	4230
query52	110	101	100	100
query53	222	262	183	183
query54	577	591	505	505
query55	85	88	87	87
query56	320	297	296	296
query57	1162	1179	1120	1120
query58	272	278	255	255
query59	2757	2747	2733	2733
query60	332	315	303	303
query61	136	124	149	124
query62	774	756	664	664
query63	218	196	182	182
query64	3078	1077	707	707
query65	4377	4293	4394	4293
query66	896	421	304	304
query67	16102	15701	15472	15472
query68	8950	888	509	509
query69	496	290	265	265
query70	1248	1090	1046	1046
query71	463	317	295	295
query72	5176	4682	4614	4614
query73	701	565	339	339
query74	8996	9135	8953	8953
query75	4185	3229	2722	2722
query76	3658	1181	742	742
query77	777	383	278	278
query78	8960	9035	8301	8301
query79	2092	927	574	574
query80	564	517	439	439
query81	473	260	226	226
query82	485	131	102	102
query83	257	296	233	233
query84	248	105	83	83
query85	828	320	264	264
query86	375	304	283	283
query87	4397	4442	4373	4373
query88	3574	2276	2231	2231
query89	400	321	286	286
query90	1769	209	212	209
query91	143	144	110	110
query92	84	63	60	60
query93	1718	979	578	578
query94	613	417	308	308
query95	373	296	279	279
query96	487	564	273	273
query97	3156	3241	3180	3180
query98	226	210	197	197
query99	1325	1388	1236	1236
Total cold run time: 275480 ms
Total hot run time: 191330 ms

doris-robot avatar Apr 15 '25 10:04 doris-robot

ClickBench: Total hot run time: 30.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2930d988d2c992f3fe1a6254e06d831d20a0a171, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.11
query3	0.26	0.19	0.19
query4	1.60	0.20	0.21
query5	0.59	0.57	0.58
query6	1.18	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.57	0.52	0.51
query10	0.56	0.56	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.59	0.60
query14	2.70	2.82	2.79
query15	0.94	0.88	0.86
query16	0.38	0.37	0.38
query17	1.02	1.02	1.00
query18	0.21	0.20	0.19
query19	1.88	1.96	1.78
query20	0.01	0.01	0.01
query21	15.35	0.97	0.59
query22	0.75	1.18	0.64
query23	15.02	1.40	0.66
query24	7.00	1.76	0.46
query25	0.45	0.07	0.21
query26	0.60	0.16	0.13
query27	0.05	0.05	0.05
query28	10.19	0.94	0.43
query29	12.58	3.96	3.27
query30	0.25	0.09	0.07
query31	2.81	0.58	0.39
query32	3.22	0.55	0.47
query33	3.11	3.00	3.00
query34	15.79	5.15	4.52
query35	4.48	4.52	4.55
query36	0.66	0.49	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.87 s
Total hot run time: 30.82 s

doris-robot avatar Apr 15 '25 10:04 doris-robot

run buildall

yujun777 avatar Apr 16 '25 04:04 yujun777

run buildall

yujun777 avatar Apr 16 '25 04:04 yujun777

run buildall

yujun777 avatar Apr 16 '25 11:04 yujun777

TPC-H: Total hot run time: 34043 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eaef1200d54db43a8b5ff63aa1975bd8d9b89cfb, data reload: false

------ Round 1 ----------------------------------
q1	26301	5141	5067	5067
q2	2068	281	192	192
q3	10397	1247	676	676
q4	10240	989	519	519
q5	8063	2473	2352	2352
q6	179	163	134	134
q7	930	749	610	610
q8	9311	1282	1151	1151
q9	6841	5155	5185	5155
q10	6897	2352	1896	1896
q11	494	292	265	265
q12	350	357	229	229
q13	17804	3669	3137	3137
q14	228	239	219	219
q15	545	483	479	479
q16	458	450	399	399
q17	579	861	352	352
q18	7535	7265	7056	7056
q19	1362	965	550	550
q20	340	321	224	224
q21	3968	3374	2404	2404
q22	1030	1003	977	977
Total cold run time: 115920 ms
Total hot run time: 34043 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5089	5059	5091	5059
q2	235	327	231	231
q3	2160	2671	2268	2268
q4	1379	1806	1447	1447
q5	4462	4398	4384	4384
q6	217	180	123	123
q7	1986	1923	1783	1783
q8	2588	2635	2547	2547
q9	7274	7259	7013	7013
q10	3041	3159	2730	2730
q11	581	532	488	488
q12	682	782	591	591
q13	3503	3927	3273	3273
q14	278	313	281	281
q15	539	492	472	472
q16	465	495	459	459
q17	1149	1562	1434	1434
q18	7813	7602	7423	7423
q19	786	763	818	763
q20	1982	2046	1843	1843
q21	5173	4930	4797	4797
q22	1133	1051	1000	1000
Total cold run time: 52515 ms
Total hot run time: 50409 ms

doris-robot avatar Apr 16 '25 12:04 doris-robot

TPC-DS: Total hot run time: 191448 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eaef1200d54db43a8b5ff63aa1975bd8d9b89cfb, data reload: false

query1	1420	1080	1061	1061
query2	6114	1773	1769	1769
query3	10998	4640	4660	4640
query4	26554	23713	23602	23602
query5	5138	617	476	476
query6	325	221	214	214
query7	3993	509	282	282
query8	312	266	243	243
query9	8502	2553	2564	2553
query10	506	317	258	258
query11	15147	15167	14840	14840
query12	159	111	105	105
query13	1554	531	394	394
query14	8835	6141	6224	6141
query15	200	196	170	170
query16	7274	624	471	471
query17	1172	780	581	581
query18	2009	422	315	315
query19	195	196	185	185
query20	131	123	125	123
query21	207	131	108	108
query22	4404	4517	4241	4241
query23	34612	33403	33735	33403
query24	8436	2401	2410	2401
query25	538	465	387	387
query26	1210	278	149	149
query27	2759	522	340	340
query28	4512	2149	2150	2149
query29	724	584	462	462
query30	280	224	196	196
query31	875	824	772	772
query32	77	66	69	66
query33	546	393	333	333
query34	837	886	540	540
query35	843	849	776	776
query36	988	1001	895	895
query37	120	102	80	80
query38	4282	4248	4180	4180
query39	1498	1461	1454	1454
query40	242	120	114	114
query41	63	54	58	54
query42	125	111	106	106
query43	497	503	460	460
query44	1314	849	806	806
query45	185	175	164	164
query46	845	1025	650	650
query47	1849	1833	1807	1807
query48	407	435	310	310
query49	770	538	439	439
query50	681	682	410	410
query51	4271	4429	4259	4259
query52	112	100	99	99
query53	224	258	193	193
query54	582	577	513	513
query55	85	81	97	81
query56	303	307	287	287
query57	1180	1180	1123	1123
query58	273	264	256	256
query59	2732	2702	2626	2626
query60	339	342	313	313
query61	134	147	127	127
query62	790	753	649	649
query63	227	191	185	185
query64	4177	1035	711	711
query65	4447	4351	4368	4351
query66	1135	402	329	329
query67	16383	15401	15320	15320
query68	8259	894	527	527
query69	517	320	281	281
query70	1113	1113	1111	1111
query71	421	320	304	304
query72	5317	4850	4631	4631
query73	628	563	347	347
query74	8900	9152	8671	8671
query75	3398	3176	2711	2711
query76	3465	1191	749	749
query77	702	375	278	278
query78	9014	9201	8378	8378
query79	1679	874	545	545
query80	567	535	433	433
query81	479	266	217	217
query82	198	126	101	101
query83	251	253	322	253
query84	247	101	85	85
query85	706	306	266	266
query86	362	319	283	283
query87	4399	4558	4357	4357
query88	3033	2178	2188	2178
query89	393	313	273	273
query90	1933	202	207	202
query91	141	150	119	119
query92	78	61	59	59
query93	1933	937	564	564
query94	658	414	311	311
query95	366	284	291	284
query96	487	560	274	274
query97	3104	3229	3083	3083
query98	239	213	205	205
query99	1334	1394	1271	1271
Total cold run time: 276979 ms
Total hot run time: 191448 ms

doris-robot avatar Apr 16 '25 12:04 doris-robot

ClickBench: Total hot run time: 29.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eaef1200d54db43a8b5ff63aa1975bd8d9b89cfb, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.12
query3	0.25	0.20	0.19
query4	1.60	0.19	0.20
query5	0.58	0.59	0.59
query6	1.19	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.57	0.52	0.51
query10	0.57	0.57	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.60	0.61
query14	1.15	1.20	1.22
query15	0.87	0.86	0.88
query16	0.39	0.38	0.38
query17	0.98	1.04	1.01
query18	0.21	0.19	0.20
query19	1.91	1.82	1.76
query20	0.02	0.01	0.02
query21	15.42	0.88	0.55
query22	0.76	1.14	0.70
query23	14.92	1.42	0.65
query24	7.42	1.60	0.33
query25	0.30	0.20	0.08
query26	0.66	0.16	0.13
query27	0.05	0.05	0.04
query28	8.64	0.89	0.43
query29	12.57	3.97	3.30
query30	0.25	0.09	0.07
query31	2.83	0.57	0.40
query32	3.23	0.54	0.46
query33	3.00	3.00	3.13
query34	15.79	5.07	4.50
query35	4.50	4.56	4.50
query36	0.68	0.51	0.48
query37	0.09	0.06	0.07
query38	0.05	0.05	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 102.94 s
Total hot run time: 29.12 s

doris-robot avatar Apr 16 '25 12:04 doris-robot

run buildall

yujun777 avatar Apr 17 '25 04:04 yujun777

TPC-H: Total hot run time: 33721 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d87aec37767855cd4ec3c2a005b87064810a0710, data reload: false

------ Round 1 ----------------------------------
q1	26426	5034	5175	5034
q2	2079	283	193	193
q3	10377	1222	717	717
q4	10236	1002	544	544
q5	7557	2370	2308	2308
q6	184	163	130	130
q7	944	750	619	619
q8	9319	1303	1103	1103
q9	6893	5010	5152	5010
q10	6842	2299	1899	1899
q11	503	283	270	270
q12	358	349	214	214
q13	17788	3676	3056	3056
q14	227	223	210	210
q15	533	482	497	482
q16	457	451	400	400
q17	583	868	350	350
q18	7629	7137	7034	7034
q19	1758	947	539	539
q20	321	326	226	226
q21	3814	3318	2409	2409
q22	1036	1004	974	974
Total cold run time: 115864 ms
Total hot run time: 33721 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5198	5176	5023	5023
q2	234	319	231	231
q3	2140	2663	2305	2305
q4	1412	1801	1394	1394
q5	4456	4373	4390	4373
q6	215	169	129	129
q7	1944	1908	1800	1800
q8	2616	2498	2497	2497
q9	7267	7265	6973	6973
q10	3010	3181	2738	2738
q11	585	518	500	500
q12	698	781	598	598
q13	3464	3908	3280	3280
q14	294	300	269	269
q15	522	481	499	481
q16	480	510	460	460
q17	1157	1556	1413	1413
q18	7834	7658	7483	7483
q19	803	854	932	854
q20	1997	1999	1802	1802
q21	5129	4802	4845	4802
q22	1103	1064	1037	1037
Total cold run time: 52558 ms
Total hot run time: 50442 ms

doris-robot avatar Apr 17 '25 04:04 doris-robot

TPC-DS: Total hot run time: 191721 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d87aec37767855cd4ec3c2a005b87064810a0710, data reload: false

query1	1413	1096	1074	1074
query2	6082	1813	1792	1792
query3	11018	4479	4693	4479
query4	25964	23944	23134	23134
query5	4844	620	485	485
query6	317	217	216	216
query7	3990	488	279	279
query8	322	247	242	242
query9	8555	2571	2591	2571
query10	490	324	260	260
query11	15918	15101	14844	14844
query12	165	112	112	112
query13	1562	521	394	394
query14	9254	6421	6264	6264
query15	207	208	178	178
query16	7628	631	495	495
query17	1091	780	603	603
query18	2069	423	309	309
query19	205	221	170	170
query20	122	126	120	120
query21	215	132	115	115
query22	4446	4655	4519	4519
query23	34811	33660	33498	33498
query24	8748	2445	2415	2415
query25	521	481	407	407
query26	1226	272	159	159
query27	2806	516	339	339
query28	4801	2184	2157	2157
query29	709	594	446	446
query30	284	228	198	198
query31	900	853	744	744
query32	77	68	69	68
query33	557	373	366	366
query34	799	880	523	523
query35	813	864	783	783
query36	963	1007	924	924
query37	115	107	78	78
query38	4241	4179	4128	4128
query39	1509	1431	1421	1421
query40	212	127	109	109
query41	55	59	57	57
query42	128	111	113	111
query43	509	503	485	485
query44	1319	821	834	821
query45	193	178	170	170
query46	844	1052	641	641
query47	1849	1885	1775	1775
query48	389	415	306	306
query49	798	557	431	431
query50	681	694	410	410
query51	4220	4323	4181	4181
query52	110	105	100	100
query53	229	262	187	187
query54	587	574	519	519
query55	82	83	84	83
query56	320	302	283	283
query57	1188	1180	1127	1127
query58	277	271	267	267
query59	2755	2804	2607	2607
query60	336	329	333	329
query61	131	130	130	130
query62	799	746	685	685
query63	229	196	198	196
query64	4183	1071	703	703
query65	4543	4339	4434	4339
query66	1080	416	302	302
query67	16010	15382	15465	15382
query68	8902	885	509	509
query69	484	309	269	269
query70	1200	1146	1120	1120
query71	468	328	312	312
query72	5657	4797	4845	4797
query73	734	674	355	355
query74	9117	9169	8699	8699
query75	3976	3199	2694	2694
query76	3711	1185	743	743
query77	795	382	283	283
query78	8997	9253	8385	8385
query79	2123	858	578	578
query80	590	517	449	449
query81	490	260	226	226
query82	190	128	103	103
query83	256	244	239	239
query84	241	109	87	87
query85	714	334	269	269
query86	372	271	282	271
query87	4448	4400	4307	4307
query88	3772	2275	2204	2204
query89	394	321	295	295
query90	1901	226	222	222
query91	141	143	112	112
query92	75	63	55	55
query93	1641	995	576	576
query94	609	406	308	308
query95	377	291	292	291
query96	493	564	277	277
query97	3107	3210	3140	3140
query98	243	207	207	207
query99	1306	1433	1294	1294
Total cold run time: 281337 ms
Total hot run time: 191721 ms

doris-robot avatar Apr 17 '25 05:04 doris-robot

ClickBench: Total hot run time: 29.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d87aec37767855cd4ec3c2a005b87064810a0710, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.25	0.20	0.19
query4	1.59	0.19	0.19
query5	0.60	0.60	0.60
query6	1.17	0.72	0.72
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.58	0.52	0.52
query10	0.57	0.56	0.58
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.59	0.60
query14	1.15	1.16	1.21
query15	0.88	0.86	0.86
query16	0.38	0.40	0.38
query17	1.04	1.05	1.03
query18	0.22	0.19	0.19
query19	1.90	1.83	1.79
query20	0.01	0.01	0.02
query21	15.39	0.88	0.55
query22	0.76	1.18	0.63
query23	14.97	1.40	0.65
query24	6.76	1.74	0.89
query25	0.48	0.07	0.15
query26	0.68	0.16	0.14
query27	0.06	0.06	0.05
query28	9.83	0.89	0.46
query29	12.51	3.97	3.30
query30	0.25	0.08	0.07
query31	2.84	0.60	0.38
query32	3.22	0.54	0.49
query33	2.99	2.98	3.12
query34	15.79	5.15	4.51
query35	4.54	4.49	4.55
query36	0.66	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 103.71 s
Total hot run time: 29.64 s

doris-robot avatar Apr 17 '25 05:04 doris-robot

run buildall

yujun777 avatar Apr 23 '25 02:04 yujun777