doris icon indicating copy to clipboard operation
doris copied to clipboard

[fix](inverted index) add in list to fast execute logic after hit index

Open zzzxl1993 opened this issue 1 year ago • 12 comments

Proposed changes

https://github.com/apache/doris/pull/35344

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

zzzxl1993 avatar May 23 '24 10:05 zzzxl1993

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar May 23 '24 10:05 doris-robot

run buildall

zzzxl1993 avatar May 23 '24 10:05 zzzxl1993

TPC-H: Total hot run time: 49982 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09b3d57e9a174a48685d8a83e17bae2c1cef62b0, data reload: false

------ Round 1 ----------------------------------
q1	17671	5046	4405	4405
q2	2083	154	142	142
q3	10515	2005	1966	1966
q4	10296	1265	1362	1265
q5	8426	3900	4021	3900
q6	237	122	124	122
q7	2058	1574	1625	1574
q8	9268	2745	2724	2724
q9	10712	10351	10172	10172
q10	8678	3533	3518	3518
q11	431	247	243	243
q12	462	309	294	294
q13	18344	3967	4021	3967
q14	355	331	328	328
q15	509	466	458	458
q16	689	578	571	571
q17	1159	951	955	951
q18	7396	6974	6859	6859
q19	1789	1658	1627	1627
q20	530	306	320	306
q21	4414	4139	4136	4136
q22	535	464	454	454
Total cold run time: 116557 ms
Total hot run time: 49982 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4317	4328	4309	4309
q2	323	239	233	233
q3	4170	4114	4127	4114
q4	2766	2762	2763	2762
q5	7278	7153	7117	7117
q6	237	120	121	120
q7	3238	2865	2824	2824
q8	4405	4515	4524	4515
q9	16996	16722	16705	16705
q10	4246	4279	4299	4279
q11	774	700	675	675
q12	1027	853	855	853
q13	7192	3726	3754	3726
q14	457	426	410	410
q15	489	462	449	449
q16	740	695	672	672
q17	3898	3852	3861	3852
q18	8786	8682	8826	8682
q19	1756	1729	1665	1665
q20	2390	2138	2151	2138
q21	8610	8443	8476	8443
q22	1096	968	1010	968
Total cold run time: 85191 ms
Total hot run time: 79511 ms

doris-robot avatar May 23 '24 11:05 doris-robot

TeamCity be ut coverage result: Function Coverage: 37.80% (8077/21370) Line Coverage: 29.45% (65952/223959) Region Coverage: 28.92% (33957/117420) Branch Coverage: 24.77% (17424/70336) Coverage Report: http://coverage.selectdb-in.cc/coverage/09b3d57e9a174a48685d8a83e17bae2c1cef62b0_09b3d57e9a174a48685d8a83e17bae2c1cef62b0/report/index.html

doris-robot avatar May 23 '24 11:05 doris-robot

TPC-DS: Total hot run time: 203741 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09b3d57e9a174a48685d8a83e17bae2c1cef62b0, data reload: false

query1	916	384	373	373
query2	6544	2828	2639	2639
query3	6929	211	206	206
query4	21134	18048	18012	18012
query5	19727	6489	6581	6489
query6	282	218	221	218
query7	4166	295	301	295
query8	436	419	420	419
query9	3112	2681	2622	2622
query10	425	289	302	289
query11	11353	10761	10723	10723
query12	122	76	75	75
query13	5584	682	729	682
query14	17647	13509	13706	13509
query15	359	232	247	232
query16	6441	272	255	255
query17	1761	1444	900	900
query18	2281	401	395	395
query19	205	147	149	147
query20	82	77	81	77
query21	193	94	94	94
query22	5298	5164	5037	5037
query23	32793	31947	32076	31947
query24	6996	6504	6527	6504
query25	507	439	410	410
query26	530	161	157	157
query27	1842	291	291	291
query28	6120	2350	2303	2303
query29	2907	2779	2868	2779
query30	244	161	166	161
query31	896	715	747	715
query32	68	63	58	58
query33	401	262	257	257
query34	846	469	471	469
query35	1136	913	936	913
query36	1285	1109	1107	1107
query37	89	58	59	58
query38	3104	2948	2943	2943
query39	1382	1317	1333	1317
query40	204	94	92	92
query41	47	42	42	42
query42	82	82	75	75
query43	787	644	612	612
query44	1139	701	705	701
query45	248	232	233	232
query46	1228	947	962	947
query47	1748	1687	1858	1687
query48	1017	704	706	704
query49	620	365	370	365
query50	853	596	616	596
query51	4751	4691	4751	4691
query52	89	74	76	74
query53	441	330	312	312
query54	2656	2450	2439	2439
query55	85	84	72	72
query56	225	210	220	210
query57	1228	1143	1064	1064
query58	208	190	199	190
query59	4193	4023	4196	4023
query60	204	193	207	193
query61	87	84	82	82
query62	866	466	485	466
query63	475	338	340	338
query64	2328	1501	1507	1501
query65	3686	3533	3560	3533
query66	799	383	375	375
query67	18265	16095	14936	14936
query68	8829	657	653	653
query69	573	352	339	339
query70	1531	1422	1614	1422
query71	414	315	308	308
query72	6478	3424	3440	3424
query73	735	329	329	329
query74	6403	5881	5886	5881
query75	5361	3672	3722	3672
query76	5407	1148	1213	1148
query77	907	253	251	251
query78	12662	11966	12008	11966
query79	7871	641	635	635
query80	1071	404	384	384
query81	497	241	235	235
query82	1173	97	97	97
query83	169	134	136	134
query84	254	69	70	69
query85	857	311	294	294
query86	332	338	290	290
query87	3252	3002	3014	3002
query88	4884	2366	2351	2351
query89	369	290	292	290
query90	1951	214	217	214
query91	165	151	135	135
query92	58	53	51	51
query93	4104	553	578	553
query94	705	209	210	209
query95	1102	1074	1051	1051
query96	657	331	331	331
query97	6487	6321	6402	6321
query98	196	180	172	172
query99	2861	921	901	901
Total cold run time: 313017 ms
Total hot run time: 203741 ms

doris-robot avatar May 23 '24 12:05 doris-robot

ClickBench: Total hot run time: 30.48 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09b3d57e9a174a48685d8a83e17bae2c1cef62b0, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.04	0.04
query4	1.81	0.07	0.06
query5	0.53	0.52	0.52
query6	1.25	0.61	0.62
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.54	0.47	0.46
query10	0.53	0.54	0.54
query11	0.13	0.08	0.08
query12	0.12	0.09	0.09
query13	0.61	0.61	0.62
query14	0.76	0.80	0.78
query15	0.78	0.78	0.77
query16	0.36	0.36	0.38
query17	0.98	1.00	0.95
query18	0.25	0.25	0.24
query19	1.92	1.87	1.87
query20	0.02	0.01	0.01
query21	15.47	0.57	0.54
query22	2.39	2.33	1.64
query23	17.47	1.09	0.96
query24	3.64	0.87	1.70
query25	0.36	0.09	0.05
query26	0.53	0.16	0.16
query27	0.04	0.04	0.04
query28	8.88	0.75	0.73
query29	12.61	2.35	2.15
query30	0.62	0.59	0.54
query31	2.80	0.39	0.37
query32	3.37	0.50	0.49
query33	3.07	3.02	3.11
query34	15.26	4.79	4.80
query35	4.84	4.83	4.82
query36	1.04	1.01	1.02
query37	0.06	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.01
Total cold run time: 103.75 s
Total hot run time: 30.48 s

doris-robot avatar May 23 '24 12:05 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 09b3d57e9a174a48685d8a83e17bae2c1cef62b0 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

doris-robot avatar May 23 '24 12:05 doris-robot

run p0

zzzxl1993 avatar May 24 '24 07:05 zzzxl1993

run buildall

zzzxl1993 avatar May 28 '24 09:05 zzzxl1993

TeamCity be ut coverage result: Function Coverage: 37.81% (8099/21421) Line Coverage: 29.46% (66128/224458) Region Coverage: 28.95% (34089/117739) Branch Coverage: 24.80% (17494/70536) Coverage Report: http://coverage.selectdb-in.cc/coverage/6882123fc1fcf58fdc0d3c37c8dd37649730bfcf_6882123fc1fcf58fdc0d3c37c8dd37649730bfcf/report/index.html

doris-robot avatar May 28 '24 10:05 doris-robot

run buildall

zzzxl1993 avatar May 30 '24 16:05 zzzxl1993

TeamCity be ut coverage result: Function Coverage: 37.81% (8099/21422) Line Coverage: 29.46% (66139/224468) Region Coverage: 28.95% (34088/117751) Branch Coverage: 24.80% (17494/70538) Coverage Report: http://coverage.selectdb-in.cc/coverage/65b51632986d26856cc36bf86256f1824bf718c2_65b51632986d26856cc36bf86256f1824bf718c2/report/index.html

doris-robot avatar May 30 '24 17:05 doris-robot

run buildall

zzzxl1993 avatar May 31 '24 08:05 zzzxl1993

TPC-H: Total hot run time: 49543 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

------ Round 1 ----------------------------------
q1	17539	4373	4322	4322
q2	2036	194	154	154
q3	10224	1888	1906	1888
q4	10364	1218	1336	1218
q5	8570	3915	3919	3915
q6	235	124	127	124
q7	2042	1607	1579	1579
q8	9543	2724	2703	2703
q9	13849	10830	10079	10079
q10	8648	3520	3514	3514
q11	413	238	254	238
q12	468	299	298	298
q13	18343	3979	4037	3979
q14	356	316	323	316
q15	508	462	463	462
q16	677	570	567	567
q17	1138	956	927	927
q18	7338	6850	6820	6820
q19	1790	1629	1601	1601
q20	543	315	326	315
q21	4438	4160	4083	4083
q22	537	441	443	441
Total cold run time: 119599 ms
Total hot run time: 49543 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4290	4328	4278	4278
q2	321	229	227	227
q3	4173	4134	4160	4134
q4	2742	2754	2765	2754
q5	7178	7100	7070	7070
q6	242	119	122	119
q7	3182	2882	2829	2829
q8	4388	4522	4546	4522
q9	16888	16866	16817	16817
q10	4248	4230	4270	4230
q11	758	702	713	702
q12	1041	870	869	869
q13	6639	3724	3728	3724
q14	473	423	428	423
q15	516	460	468	460
q16	732	674	671	671
q17	3793	3817	3822	3817
q18	8789	8680	8797	8680
q19	1723	1688	1676	1676
q20	2384	2123	2143	2123
q21	8423	8516	8522	8516
q22	1037	955	949	949
Total cold run time: 83960 ms
Total hot run time: 79590 ms

doris-robot avatar May 31 '24 09:05 doris-robot

TPC-DS: Total hot run time: 203770 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

query1	925	391	414	391
query2	6519	2770	2627	2627
query3	6935	215	215	215
query4	20580	17936	18015	17936
query5	19715	6474	6556	6474
query6	288	221	226	221
query7	4154	303	310	303
query8	428	423	393	393
query9	3089	2674	2608	2608
query10	428	302	310	302
query11	11407	10793	10725	10725
query12	116	85	72	72
query13	5591	677	677	677
query14	18073	13363	13768	13363
query15	390	243	256	243
query16	6476	292	273	273
query17	1690	1447	875	875
query18	2330	409	409	409
query19	216	151	153	151
query20	83	81	81	81
query21	198	92	95	92
query22	5357	5010	5094	5010
query23	32514	31865	31974	31865
query24	6933	6561	6510	6510
query25	523	425	415	415
query26	637	157	160	157
query27	2062	291	299	291
query28	6197	2376	2319	2319
query29	2856	2657	2710	2657
query30	242	165	168	165
query31	915	767	736	736
query32	66	63	58	58
query33	408	264	243	243
query34	863	470	473	470
query35	1119	952	934	934
query36	1239	1410	1249	1249
query37	91	62	61	61
query38	3038	2946	2934	2934
query39	1378	1324	1316	1316
query40	311	94	93	93
query41	46	44	43	43
query42	87	86	93	86
query43	821	733	776	733
query44	1117	722	720	720
query45	251	235	238	235
query46	1240	969	969	969
query47	2013	1851	1741	1741
query48	1038	697	715	697
query49	636	374	364	364
query50	877	596	577	577
query51	4703	4685	4645	4645
query52	96	84	85	84
query53	447	323	318	318
query54	2655	2467	2502	2467
query55	100	81	89	81
query56	245	204	202	202
query57	1230	1091	1286	1091
query58	218	212	204	204
query59	4239	4045	3870	3870
query60	211	207	219	207
query61	98	95	96	95
query62	820	559	568	559
query63	493	338	341	338
query64	2699	1550	1458	1458
query65	3657	3515	3567	3515
query66	809	385	375	375
query67	16551	15634	16090	15634
query68	9329	671	663	663
query69	585	371	357	357
query70	1528	1515	1313	1313
query71	416	317	322	317
query72	6530	3519	3531	3519
query73	753	337	328	328
query74	6308	5798	5839	5798
query75	5245	3610	3678	3610
query76	5787	1131	1128	1128
query77	977	262	262	262
query78	12485	11635	11430	11430
query79	8399	651	644	644
query80	1488	401	408	401
query81	496	240	235	235
query82	1605	98	98	98
query83	180	132	140	132
query84	267	71	72	71
query85	882	319	321	319
query86	353	293	335	293
query87	3255	2998	3063	2998
query88	4903	2370	2353	2353
query89	385	295	289	289
query90	1959	190	210	190
query91	177	141	144	141
query92	58	54	56	54
query93	6631	601	558	558
query94	704	213	207	207
query95	1109	1059	1061	1059
query96	660	346	336	336
query97	6486	6428	6462	6428
query98	177	170	171	170
query99	2946	847	887	847
Total cold run time: 316840 ms
Total hot run time: 203770 ms

doris-robot avatar May 31 '24 09:05 doris-robot

TeamCity be ut coverage result: Function Coverage: 37.81% (8099/21422) Line Coverage: 29.47% (66143/224467) Region Coverage: 28.95% (34095/117752) Branch Coverage: 24.81% (17498/70534) Coverage Report: http://coverage.selectdb-in.cc/coverage/ffe04246780f48f086ac166b2304da4fd80e4a69_ffe04246780f48f086ac166b2304da4fd80e4a69/report/index.html

doris-robot avatar May 31 '24 09:05 doris-robot

ClickBench: Total hot run time: 30.48 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.26	0.04	0.04
query4	1.77	0.08	0.06
query5	0.54	0.52	0.52
query6	1.26	0.63	0.61
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.52	0.47	0.47
query10	0.53	0.53	0.54
query11	0.11	0.08	0.08
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.80	0.81	0.78
query15	0.78	0.76	0.77
query16	0.35	0.37	0.36
query17	1.02	1.00	1.00
query18	0.23	0.24	0.26
query19	1.92	1.82	1.86
query20	0.01	0.00	0.00
query21	15.48	0.58	0.55
query22	2.35	2.84	1.77
query23	17.38	0.94	0.95
query24	6.51	0.63	3.31
query25	0.37	0.09	0.05
query26	0.91	0.16	0.16
query27	0.04	0.04	0.05
query28	4.72	0.71	0.76
query29	12.72	2.26	2.31
query30	0.61	0.53	0.53
query31	2.82	0.39	0.37
query32	3.36	0.50	0.50
query33	3.07	3.07	3.12
query34	15.26	4.80	4.80
query35	4.85	4.82	4.83
query36	1.07	1.02	1.00
query37	0.06	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.02	0.01
query43	0.03	0.02	0.01
Total cold run time: 102.9 s
Total hot run time: 30.48 s

doris-robot avatar May 31 '24 09:05 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

doris-robot avatar May 31 '24 09:05 doris-robot

run buildall

xiaokang avatar Jun 01 '24 00:06 xiaokang

TPC-H: Total hot run time: 49852 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

------ Round 1 ----------------------------------
q1	18074	4436	4349	4349
q2	2036	152	184	152
q3	10401	1896	1930	1896
q4	10251	1274	1326	1274
q5	8808	3917	3920	3917
q6	232	125	127	125
q7	2048	1677	1583	1583
q8	9290	2748	2705	2705
q9	10623	10350	10326	10326
q10	8623	3480	3482	3480
q11	429	236	248	236
q12	470	315	298	298
q13	18388	3962	4028	3962
q14	355	326	324	324
q15	504	463	458	458
q16	674	574	577	574
q17	1134	972	983	972
q18	7247	6889	6842	6842
q19	1814	1668	1561	1561
q20	531	286	300	286
q21	4450	4105	4091	4091
q22	526	446	441	441
Total cold run time: 116908 ms
Total hot run time: 49852 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4306	4282	4282
q2	321	230	223	223
q3	4200	4160	4114	4114
q4	2765	2741	2743	2741
q5	7179	7125	7046	7046
q6	240	121	119	119
q7	3238	2771	2793	2771
q8	4302	4424	4487	4424
q9	16911	16877	16785	16785
q10	4277	4252	4273	4252
q11	735	705	656	656
q12	1023	863	825	825
q13	7343	3703	3747	3703
q14	453	432	427	427
q15	548	448	453	448
q16	760	679	670	670
q17	3844	3876	3785	3785
q18	8816	8721	8783	8721
q19	1710	1726	1684	1684
q20	2381	2131	2123	2123
q21	8503	8443	8390	8390
q22	1030	960	944	944
Total cold run time: 84904 ms
Total hot run time: 79133 ms

doris-robot avatar Jun 01 '24 01:06 doris-robot

TeamCity be ut coverage result: Function Coverage: 37.80% (8099/21424) Line Coverage: 29.47% (66144/224476) Region Coverage: 28.95% (34093/117763) Branch Coverage: 24.80% (17495/70546) Coverage Report: http://coverage.selectdb-in.cc/coverage/ffe04246780f48f086ac166b2304da4fd80e4a69_ffe04246780f48f086ac166b2304da4fd80e4a69/report/index.html

doris-robot avatar Jun 01 '24 01:06 doris-robot

TPC-DS: Total hot run time: 203674 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

query1	928	397	385	385
query2	6553	2838	2627	2627
query3	6920	210	204	204
query4	21814	18162	18120	18120
query5	19734	6521	6578	6521
query6	281	220	212	212
query7	4155	312	334	312
query8	448	444	420	420
query9	3166	2731	2649	2649
query10	410	304	303	303
query11	11360	10695	10594	10594
query12	126	75	72	72
query13	5605	730	706	706
query14	17494	13502	13256	13256
query15	365	232	246	232
query16	6462	299	271	271
query17	1714	1421	890	890
query18	2306	417	409	409
query19	219	146	151	146
query20	77	75	82	75
query21	195	96	95	95
query22	5187	5041	5069	5041
query23	32672	32081	31869	31869
query24	7057	6548	6570	6548
query25	511	436	429	429
query26	535	162	169	162
query27	1889	299	294	294
query28	6113	2381	2350	2350
query29	2981	2857	2659	2659
query30	241	166	169	166
query31	916	738	746	738
query32	73	69	63	63
query33	402	264	255	255
query34	838	467	471	467
query35	1106	855	876	855
query36	1282	1137	1337	1137
query37	90	64	61	61
query38	3071	2904	2890	2890
query39	1392	1326	1323	1323
query40	202	99	96	96
query41	46	46	42	42
query42	91	87	80	80
query43	746	721	753	721
query44	1120	716	716	716
query45	253	235	233	233
query46	1213	958	958	958
query47	2015	1987	1697	1697
query48	1035	738	710	710
query49	620	377	370	370
query50	851	604	640	604
query51	4754	4682	4649	4649
query52	89	82	82	82
query53	456	329	315	315
query54	2655	2489	2479	2479
query55	92	83	87	83
query56	227	209	214	209
query57	1227	1084	1119	1084
query58	202	206	200	200
query59	3937	3958	3873	3873
query60	207	198	193	193
query61	98	96	97	96
query62	841	479	477	477
query63	485	341	360	341
query64	2569	1583	1472	1472
query65	3639	3594	3541	3541
query66	767	383	406	383
query67	15787	15703	15370	15370
query68	8931	646	673	646
query69	572	377	356	356
query70	1664	1420	1636	1420
query71	426	319	312	312
query72	6573	3496	3459	3459
query73	744	322	326	322
query74	6240	5813	5855	5813
query75	5397	3741	3664	3664
query76	5563	1132	1180	1132
query77	968	258	262	258
query78	12557	12046	11674	11674
query79	7673	654	648	648
query80	1098	409	409	409
query81	498	233	237	233
query82	1652	100	98	98
query83	176	129	129	129
query84	262	70	72	70
query85	883	335	323	323
query86	333	306	297	297
query87	3201	2979	3035	2979
query88	4915	2387	2380	2380
query89	387	286	299	286
query90	1954	220	216	216
query91	178	163	163	163
query92	61	53	56	53
query93	5467	545	563	545
query94	703	214	217	214
query95	1116	1037	1051	1037
query96	653	348	323	323
query97	6438	6338	6416	6338
query98	184	172	174	172
query99	2892	896	825	825
Total cold run time: 313271 ms
Total hot run time: 203674 ms

doris-robot avatar Jun 01 '24 01:06 doris-robot

ClickBench: Total hot run time: 30.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.02
query3	0.25	0.05	0.05
query4	1.77	0.05	0.06
query5	0.53	0.52	0.52
query6	1.28	0.61	0.65
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.54	0.49	0.48
query10	0.53	0.54	0.55
query11	0.12	0.09	0.09
query12	0.11	0.08	0.09
query13	0.63	0.61	0.60
query14	0.78	0.77	0.79
query15	0.79	0.75	0.77
query16	0.36	0.39	0.39
query17	1.02	0.99	1.04
query18	0.24	0.23	0.25
query19	1.94	1.85	1.82
query20	0.02	0.01	0.01
query21	15.84	0.56	0.54
query22	1.98	2.32	1.61
query23	17.20	1.11	0.92
query24	7.79	0.90	0.98
query25	0.36	0.12	0.05
query26	0.81	0.16	0.16
query27	0.05	0.04	0.03
query28	5.51	0.74	0.71
query29	12.59	2.32	2.26
query30	0.61	0.53	0.53
query31	2.80	0.39	0.37
query32	3.40	0.50	0.49
query33	3.09	3.08	3.07
query34	15.30	4.77	4.80
query35	4.83	4.81	4.85
query36	1.06	1.01	1.01
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.64 s
Total hot run time: 30.54 s

doris-robot avatar Jun 01 '24 01:06 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ffe04246780f48f086ac166b2304da4fd80e4a69 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

doris-robot avatar Jun 01 '24 01:06 doris-robot

run buildall

zzzxl1993 avatar Jun 02 '24 06:06 zzzxl1993

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Jun 02 '24 06:06 github-actions[bot]

TPC-H: Total hot run time: 49994 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5f5273383dc69cec51e4698beb22ef9b389aa14, data reload: false

------ Round 1 ----------------------------------
q1	18161	4464	4347	4347
q2	2071	160	150	150
q3	10457	1925	1908	1908
q4	10323	1272	1325	1272
q5	8428	3931	3974	3931
q6	239	150	124	124
q7	2039	1624	1623	1623
q8	9284	2745	2711	2711
q9	10675	10255	10419	10255
q10	8672	3533	3479	3479
q11	429	242	254	242
q12	474	306	305	305
q13	18353	4001	4062	4001
q14	358	322	325	322
q15	514	456	463	456
q16	685	576	569	569
q17	1161	980	992	980
q18	7272	6910	6820	6820
q19	1807	1644	1632	1632
q20	545	328	313	313
q21	4502	4117	4133	4117
q22	533	437	457	437
Total cold run time: 116982 ms
Total hot run time: 49994 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4382	4294	4294
q2	330	226	226	226
q3	4165	4158	4182	4158
q4	2757	2740	2753	2740
q5	7228	7111	7075	7075
q6	237	124	122	122
q7	3234	2784	2761	2761
q8	4351	4453	4514	4453
q9	16885	16922	16739	16739
q10	4227	4272	4210	4210
q11	748	701	672	672
q12	1033	854	864	854
q13	6653	3714	3717	3714
q14	450	421	421	421
q15	509	451	449	449
q16	728	683	690	683
q17	3807	3828	3885	3828
q18	8845	8871	8754	8754
q19	1744	1751	1653	1653
q20	2354	2142	2171	2142
q21	8492	8419	8571	8419
q22	1040	976	983	976
Total cold run time: 84202 ms
Total hot run time: 79343 ms

doris-robot avatar Jun 02 '24 07:06 doris-robot

TeamCity be ut coverage result: Function Coverage: 37.80% (8099/21424) Line Coverage: 29.47% (66147/224477) Region Coverage: 28.96% (34100/117763) Branch Coverage: 24.81% (17505/70546) Coverage Report: http://coverage.selectdb-in.cc/coverage/e5f5273383dc69cec51e4698beb22ef9b389aa14_e5f5273383dc69cec51e4698beb22ef9b389aa14/report/index.html

doris-robot avatar Jun 02 '24 07:06 doris-robot

TPC-DS: Total hot run time: 204624 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5f5273383dc69cec51e4698beb22ef9b389aa14, data reload: false

query1	939	393	433	393
query2	6524	2811	2671	2671
query3	6919	215	208	208
query4	20849	18090	17872	17872
query5	19727	6514	6541	6514
query6	284	224	230	224
query7	4156	300	296	296
query8	407	392	415	392
query9	3141	2713	2639	2639
query10	424	294	293	293
query11	11364	10669	10711	10669
query12	120	74	77	74
query13	5607	709	677	677
query14	17622	13707	13604	13604
query15	367	247	242	242
query16	6463	292	263	263
query17	1699	1456	857	857
query18	2337	412	402	402
query19	209	147	157	147
query20	82	79	84	79
query21	190	98	97	97
query22	5063	5162	4999	4999
query23	32625	32080	31947	31947
query24	7011	6486	6513	6486
query25	532	438	410	410
query26	530	161	163	161
query27	1879	298	297	297
query28	6253	2397	2347	2347
query29	2858	2723	2709	2709
query30	248	166	166	166
query31	912	745	737	737
query32	70	72	59	59
query33	399	267	255	255
query34	833	472	465	465
query35	1149	950	944	944
query36	1328	1259	1237	1237
query37	89	62	62	62
query38	3045	2997	2933	2933
query39	1370	1322	1294	1294
query40	211	94	98	94
query41	48	45	44	44
query42	93	97	87	87
query43	818	676	745	676
query44	1129	720	722	720
query45	249	237	239	237
query46	1227	983	990	983
query47	1838	1861	1867	1861
query48	1012	732	703	703
query49	625	386	373	373
query50	869	652	613	613
query51	4746	4642	4613	4613
query52	94	92	80	80
query53	469	325	324	324
query54	2689	2432	2504	2432
query55	97	83	94	83
query56	231	226	197	197
query57	1136	1096	1176	1096
query58	221	197	205	197
query59	4214	4013	3834	3834
query60	217	212	213	212
query61	98	96	95	95
query62	847	453	513	453
query63	494	345	350	345
query64	2559	1579	1431	1431
query65	3634	3522	3511	3511
query66	779	382	365	365
query67	16137	15193	15185	15185
query68	9804	660	642	642
query69	564	371	342	342
query70	1962	1427	1406	1406
query71	425	309	329	309
query72	6683	3481	3481	3481
query73	740	325	325	325
query74	6351	5840	5806	5806
query75	5524	3622	3731	3622
query76	6119	1104	1138	1104
query77	1051	258	253	253
query78	12644	12694	12565	12565
query79	12087	640	636	636
query80	746	396	397	396
query81	493	236	231	231
query82	960	99	98	98
query83	197	135	140	135
query84	259	71	70	70
query85	776	335	317	317
query86	340	298	330	298
query87	3238	3057	3080	3057
query88	5557	2362	2365	2362
query89	415	309	285	285
query90	2186	210	211	210
query91	182	142	140	140
query92	61	54	55	54
query93	6189	572	566	566
query94	896	213	203	203
query95	1074	1062	1051	1051
query96	644	334	337	334
query97	6376	6329	6386	6329
query98	198	178	174	174
query99	2772	916	858	858
Total cold run time: 319918 ms
Total hot run time: 204624 ms

doris-robot avatar Jun 02 '24 07:06 doris-robot

ClickBench: Total hot run time: 31.07 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5f5273383dc69cec51e4698beb22ef9b389aa14, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.04
query4	1.80	0.06	0.06
query5	0.54	0.52	0.52
query6	1.26	0.62	0.63
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.53	0.50	0.49
query10	0.55	0.54	0.54
query11	0.12	0.08	0.09
query12	0.12	0.09	0.10
query13	0.62	0.61	0.60
query14	0.80	0.78	0.79
query15	0.78	0.76	0.77
query16	0.37	0.37	0.37
query17	1.03	1.03	1.00
query18	0.24	0.24	0.22
query19	1.93	1.83	1.81
query20	0.01	0.02	0.01
query21	15.45	0.57	0.55
query22	1.96	1.82	1.54
query23	16.80	0.84	0.92
query24	5.48	1.56	1.42
query25	0.39	0.07	0.06
query26	0.70	0.15	0.15
query27	0.04	0.03	0.04
query28	6.99	0.74	0.74
query29	12.60	2.27	2.30
query30	0.60	0.54	0.54
query31	2.81	0.39	0.39
query32	3.37	0.50	0.50
query33	3.05	3.09	3.08
query34	15.27	4.80	4.80
query35	4.90	4.83	4.82
query36	1.07	1.03	1.02
query37	0.05	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.15	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 102.94 s
Total hot run time: 31.07 s

doris-robot avatar Jun 02 '24 07:06 doris-robot