doris icon indicating copy to clipboard operation
doris copied to clipboard

[chore](Nereids) more reasonable parse select list only query

Open morrySnow opened this issue 1 year ago • 9 comments

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

morrySnow avatar Feb 23 '24 11:02 morrySnow

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

doris-robot avatar Feb 23 '24 11:02 doris-robot

run buildall

morrySnow avatar Feb 23 '24 11:02 morrySnow

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Feb 23 '24 14:02 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Feb 23 '24 14:02 github-actions[bot]

run buildall

yiguolei avatar Feb 24 '24 03:02 yiguolei

TPC-H: Total hot run time: 40883 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16d155342ce80c1822c51a536774ff7db648c6d5, data reload: false

------ Round 1 ----------------------------------
q1	17724	4930	4859	4859
q2	2043	145	134	134
q3	10575	972	970	970
q4	4654	953	954	953
q5	7587	3113	3175	3113
q6	195	138	134	134
q7	1261	766	755	755
q8	9256	2055	2036	2036
q9	7448	6572	6552	6552
q10	8293	2656	2655	2655
q11	416	204	214	204
q12	718	330	333	330
q13	17924	3641	3599	3599
q14	293	251	252	251
q15	588	513	504	504
q16	467	403	409	403
q17	917	871	853	853
q18	7457	6700	6610	6610
q19	1552	1495	1488	1488
q20	529	270	268	268
q21	6396	3886	3956	3886
q22	859	342	326	326
Total cold run time: 107152 ms
Total hot run time: 40883 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4829	4816	4850	4816
q2	295	179	190	179
q3	3606	3584	3586	3584
q4	2538	2538	2524	2524
q5	5757	5743	5730	5730
q6	210	127	128	127
q7	2250	1690	1671	1671
q8	2990	3043	3078	3043
q9	8678	8627	8640	8627
q10	6769	4230	4248	4230
q11	494	383	360	360
q12	767	560	547	547
q13	4395	3430	3378	3378
q14	264	249	240	240
q15	629	524	495	495
q16	488	464	431	431
q17	1653	1580	1570	1570
q18	8264	7688	7690	7688
q19	1641	1615	1630	1615
q20	2094	1833	1828	1828
q21	6562	6189	6205	6189
q22	581	505	512	505
Total cold run time: 65754 ms
Total hot run time: 59377 ms

doris-robot avatar Feb 24 '24 04:02 doris-robot

TPC-DS: Total hot run time: 177201 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16d155342ce80c1822c51a536774ff7db648c6d5, data reload: false

query1	918	352	350	350
query2	6518	1830	1779	1779
query3	6700	212	199	199
query4	23058	21213	21192	21192
query5	4274	376	373	373
query6	255	181	172	172
query7	4615	297	297	297
query8	255	194	219	194
query9	8418	2832	2821	2821
query10	419	230	226	226
query11	14899	14467	14556	14467
query12	148	83	85	83
query13	1688	420	416	416
query14	9226	7692	7732	7692
query15	223	187	192	187
query16	7506	256	252	252
query17	1412	562	534	534
query18	1948	264	269	264
query19	203	148	155	148
query20	89	87	85	85
query21	190	120	126	120
query22	4845	4789	4716	4716
query23	32375	31557	31533	31533
query24	12820	3417	3417	3417
query25	651	360	368	360
query26	1889	159	170	159
query27	3026	317	308	308
query28	6600	1809	1810	1809
query29	1158	633	638	633
query30	276	137	146	137
query31	932	749	761	749
query32	95	59	61	59
query33	721	240	240	240
query34	1064	498	499	498
query35	943	815	844	815
query36	1039	922	872	872
query37	191	65	60	60
query38	3282	3152	3215	3152
query39	1374	1342	1349	1342
query40	281	110	109	109
query41	37	38	36	36
query42	110	100	98	98
query43	479	449	456	449
query44	1049	684	693	684
query45	200	182	176	176
query46	1027	761	771	761
query47	1646	1577	1530	1530
query48	420	345	344	344
query49	1244	310	305	305
query50	763	373	386	373
query51	4463	4308	4332	4308
query52	109	99	93	93
query53	398	302	307	302
query54	289	227	224	224
query55	84	80	80	80
query56	224	205	212	205
query57	1042	912	974	912
query58	206	194	204	194
query59	2347	2190	2182	2182
query60	253	218	224	218
query61	85	83	88	83
query62	599	404	390	390
query63	333	297	302	297
query64	6265	3036	3112	3036
query65	3278	3273	3247	3247
query66	1351	325	322	322
query67	14608	14463	14414	14414
query68	5050	547	536	536
query69	519	358	364	358
query70	1273	1184	1231	1184
query71	440	252	258	252
query72	6365	2793	2627	2627
query73	699	310	310	310
query74	6941	6488	6448	6448
query75	3215	2608	2557	2557
query76	3280	1129	1218	1129
query77	358	241	243	241
query78	9426	8809	8745	8745
query79	998	506	506	506
query80	509	356	352	352
query81	437	214	206	206
query82	170	86	87	86
query83	141	119	132	119
query84	221	81	80	80
query85	1035	345	345	345
query86	293	305	313	305
query87	3465	3337	3302	3302
query88	2711	2300	2292	2292
query89	470	369	358	358
query90	1929	165	166	165
query91	157	127	126	126
query92	58	48	48	48
query93	1016	520	511	511
query94	1112	178	174	174
query95	437	346	340	340
query96	572	261	267	261
query97	4474	4271	4226	4226
query98	228	210	203	203
query99	1090	734	727	727
Total cold run time: 268522 ms
Total hot run time: 177201 ms

doris-robot avatar Feb 24 '24 04:02 doris-robot

ClickBench: Total hot run time: 31.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16d155342ce80c1822c51a536774ff7db648c6d5, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.08
query4	1.67	0.09	0.09
query5	0.48	0.47	0.48
query6	1.36	0.62	0.62
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.45	0.46
query10	0.50	0.50	0.50
query11	0.13	0.09	0.10
query12	0.12	0.10	0.10
query13	0.60	0.59	0.59
query14	0.76	0.78	0.79
query15	0.82	0.79	0.79
query16	0.32	0.33	0.35
query17	0.93	0.89	0.92
query18	0.18	0.14	0.19
query19	1.74	1.70	1.66
query20	0.01	0.02	0.01
query21	15.41	0.62	0.59
query22	2.89	5.09	2.89
query23	17.09	1.05	0.95
query24	2.07	0.53	0.38
query25	0.61	0.07	0.05
query26	0.16	0.13	0.14
query27	0.05	0.05	0.05
query28	12.06	0.85	0.82
query29	12.53	3.34	3.33
query30	0.55	0.53	0.48
query31	2.77	0.37	0.37
query32	3.33	0.48	0.47
query33	3.12	3.17	3.17
query34	15.70	4.49	4.50
query35	4.50	4.50	4.47
query36	1.08	0.98	0.97
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.17	0.15	0.13
query41	0.06	0.02	0.02
query42	0.03	0.02	0.02
query43	0.02	0.02	0.01
Total cold run time: 104.87 s
Total hot run time: 31.54 s

doris-robot avatar Feb 24 '24 04:02 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 16d155342ce80c1822c51a536774ff7db648c6d5 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       15.4 seconds inserted 10000000 Rows, about 649K ops/s

doris-robot avatar Feb 24 '24 04:02 doris-robot

run buildall

morrySnow avatar Feb 26 '24 08:02 morrySnow

TPC-H: Total hot run time: 38143 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 07abb8736573c6d9eb1e71c1d725680584630fa4, data reload: false

------ Round 1 ----------------------------------
q1	18961	4121	4125	4121
q2	2726	146	139	139
q3	11752	964	971	964
q4	4723	977	965	965
q5	8272	3021	3104	3021
q6	189	127	129	127
q7	1279	798	748	748
q8	9404	2047	2013	2013
q9	7232	6330	6282	6282
q10	8170	2486	2509	2486
q11	418	214	214	214
q12	746	305	315	305
q13	17939	3216	3223	3216
q14	275	247	254	247
q15	531	505	499	499
q16	488	422	422	422
q17	933	872	878	872
q18	6534	6024	5926	5926
q19	1569	1514	1506	1506
q20	487	276	271	271
q21	5974	3551	3527	3527
q22	412	300	272	272
Total cold run time: 109014 ms
Total hot run time: 38143 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4067	4006	4049	4006
q2	306	214	216	214
q3	2973	2971	2975	2971
q4	1888	1847	1840	1840
q5	5160	5128	5161	5128
q6	199	115	117	115
q7	2156	1738	1717	1717
q8	3148	3184	3222	3184
q9	8282	8245	8263	8245
q10	3671	3689	3647	3647
q11	524	426	434	426
q12	690	520	548	520
q13	3750	3099	3060	3060
q14	277	249	254	249
q15	539	498	506	498
q16	515	463	492	463
q17	1709	1695	1671	1671
q18	7819	7544	7355	7355
q19	1667	1645	1628	1628
q20	2086	1903	1895	1895
q21	4833	4612	4718	4612
q22	498	444	443	443
Total cold run time: 56757 ms
Total hot run time: 53887 ms

doris-robot avatar Feb 26 '24 09:02 doris-robot

TPC-DS: Total hot run time: 168673 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 07abb8736573c6d9eb1e71c1d725680584630fa4, data reload: false

query1	934	341	325	325
query2	6542	1693	1662	1662
query3	6707	201	205	201
query4	23153	20503	20510	20503
query5	4367	376	366	366
query6	277	164	173	164
query7	4613	314	307	307
query8	258	195	194	194
query9	8558	2216	2205	2205
query10	421	224	224	224
query11	14486	14231	14043	14043
query12	139	83	81	81
query13	1649	425	433	425
query14	8301	6336	6616	6336
query15	216	171	172	171
query16	7093	262	263	262
query17	1022	522	502	502
query18	1901	277	261	261
query19	187	153	155	153
query20	84	82	84	82
query21	201	120	129	120
query22	4679	4522	4482	4482
query23	30701	29717	30008	29717
query24	11590	3106	3147	3106
query25	624	366	349	349
query26	1707	152	171	152
query27	3045	315	317	315
query28	7055	1803	1791	1791
query29	1131	565	565	565
query30	275	132	135	132
query31	855	687	677	677
query32	95	60	56	56
query33	721	234	225	225
query34	1046	491	501	491
query35	851	760	751	751
query36	999	905	825	825
query37	131	62	61	61
query38	3095	2976	2970	2970
query39	1296	1263	1251	1251
query40	284	105	106	105
query41	41	36	35	35
query42	106	100	102	100
query43	448	434	420	420
query44	1076	697	713	697
query45	190	174	170	170
query46	1041	812	816	812
query47	1520	1436	1359	1359
query48	427	368	367	367
query49	1110	304	289	289
query50	776	380	391	380
query51	4337	4291	4286	4286
query52	101	90	93	90
query53	340	266	265	265
query54	314	239	236	236
query55	90	79	82	79
query56	220	205	212	205
query57	960	866	895	866
query58	208	193	203	193
query59	2188	2030	2071	2030
query60	237	228	224	224
query61	85	85	84	84
query62	599	344	361	344
query63	286	265	266	265
query64	6611	2993	3125	2993
query65	3244	3201	3208	3201
query66	1419	320	313	313
query67	14461	13975	14018	13975
query68	4964	589	574	574
query69	536	378	398	378
query70	1240	1230	1224	1224
query71	357	260	249	249
query72	6238	2818	2657	2657
query73	714	333	326	326
query74	6344	5948	5985	5948
query75	3072	2490	2416	2416
query76	2796	1014	1152	1014
query77	344	240	233	233
query78	8980	8408	8530	8408
query79	951	525	539	525
query80	502	356	366	356
query81	440	201	197	197
query82	159	86	88	86
query83	182	122	119	119
query84	270	81	80	80
query85	1047	344	333	333
query86	297	295	276	276
query87	3273	3073	3079	3073
query88	2797	2401	2426	2401
query89	386	333	343	333
query90	1789	169	171	169
query91	149	125	136	125
query92	55	47	49	47
query93	1023	525	511	511
query94	1067	178	177	177
query95	441	343	331	331
query96	571	272	274	272
query97	4226	4117	4138	4117
query98	222	210	191	191
query99	1035	674	641	641
Total cold run time: 260191 ms
Total hot run time: 168673 ms

doris-robot avatar Feb 26 '24 09:02 doris-robot

ClickBench: Total hot run time: 32.26 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 07abb8736573c6d9eb1e71c1d725680584630fa4, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.02	0.02
query3	0.22	0.06	0.06
query4	1.67	0.10	0.10
query5	0.52	0.52	0.52
query6	1.27	0.64	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.59	0.58	0.56
query11	0.13	0.10	0.09
query12	0.13	0.10	0.10
query13	0.62	0.62	0.61
query14	0.78	0.81	0.80
query15	0.83	0.82	0.81
query16	0.39	0.38	0.36
query17	1.01	0.96	0.99
query18	0.25	0.27	0.26
query19	1.77	1.74	1.76
query20	0.02	0.01	0.01
query21	15.42	0.64	0.62
query22	3.10	4.41	2.63
query23	17.59	1.02	0.89
query24	2.01	0.37	0.23
query25	0.40	0.14	0.08
query26	0.17	0.15	0.15
query27	0.04	0.06	0.04
query28	12.47	0.87	0.82
query29	12.67	3.44	3.31
query30	0.58	0.48	0.53
query31	2.78	0.35	0.35
query32	3.35	0.48	0.49
query33	3.17	3.25	3.22
query34	15.37	4.73	4.74
query35	4.71	4.76	4.71
query36	1.13	1.08	1.06
query37	0.06	0.04	0.04
query38	0.04	0.03	0.02
query39	0.02	0.01	0.02
query40	0.18	0.13	0.13
query41	0.06	0.02	0.02
query42	0.03	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.3 s
Total hot run time: 32.26 s

doris-robot avatar Feb 26 '24 09:02 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 07abb8736573c6d9eb1e71c1d725680584630fa4 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.0 seconds inserted 10000000 Rows, about 588K ops/s

doris-robot avatar Feb 26 '24 09:02 doris-robot

run p0

morrySnow avatar Feb 26 '24 10:02 morrySnow

run buildall

morrySnow avatar Feb 26 '24 12:02 morrySnow

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Feb 27 '24 02:02 github-actions[bot]