doris icon indicating copy to clipboard operation
doris copied to clipboard

[opt](fe) Adjust two default configuration for fe

Open swjtu-zhanglei opened this issue 1 year ago • 8 comments

  • replace CMS gc alogrithem with G1
  • set label keep time smaller by default

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

swjtu-zhanglei avatar May 23 '24 07:05 swjtu-zhanglei

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar May 23 '24 07:05 doris-robot

run buildall

dataroaring avatar May 23 '24 08:05 dataroaring

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar May 23 '24 08:05 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar May 23 '24 08:05 github-actions[bot]

TPC-H: Total hot run time: 49346 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4fecbcb41efbe83863eac7ec83aa200cbed78388, data reload: false

------ Round 1 ----------------------------------
q1	17916	4307	4314	4307
q2	2030	188	143	143
q3	10400	1918	1941	1918
q4	10341	1229	1303	1229
q5	8726	3907	3871	3871
q6	229	128	127	127
q7	2058	1585	1611	1585
q8	9274	2713	2705	2705
q9	10584	10261	10134	10134
q10	8616	3521	3476	3476
q11	437	241	244	241
q12	460	302	306	302
q13	18331	3940	4047	3940
q14	359	329	324	324
q15	501	461	448	448
q16	693	566	572	566
q17	1121	953	961	953
q18	7267	6803	6815	6803
q19	1738	1588	1579	1579
q20	515	294	313	294
q21	4375	4131	4005	4005
q22	502	396	403	396
Total cold run time: 116473 ms
Total hot run time: 49346 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4301	4290	4284	4284
q2	320	231	230	230
q3	4170	4194	4156	4156
q4	2739	2760	2716	2716
q5	7114	7215	7219	7215
q6	237	122	123	122
q7	3297	2879	2871	2871
q8	4374	4475	4480	4475
q9	16862	16796	16802	16796
q10	4234	4271	4245	4245
q11	790	676	694	676
q12	1020	841	845	841
q13	6793	3764	3755	3755
q14	443	425	426	425
q15	508	451	459	451
q16	740	680	672	672
q17	3912	3827	3868	3827
q18	8715	8712	8776	8712
q19	1710	1669	1633	1633
q20	2377	2109	2105	2105
q21	8413	8561	8530	8530
q22	1031	934	944	934
Total cold run time: 84100 ms
Total hot run time: 79671 ms

doris-robot avatar May 23 '24 09:05 doris-robot

TPC-DS: Total hot run time: 203822 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4fecbcb41efbe83863eac7ec83aa200cbed78388, data reload: false

query1	938	396	409	396
query2	6550	2877	2603	2603
query3	6916	211	211	211
query4	20339	18091	18085	18085
query5	19732	6591	6638	6591
query6	300	225	247	225
query7	4164	296	315	296
query8	279	247	250	247
query9	3117	2689	2633	2633
query10	420	315	308	308
query11	11376	10741	10702	10702
query12	120	82	75	75
query13	5599	706	691	691
query14	17759	13635	13384	13384
query15	371	227	227	227
query16	6436	286	261	261
query17	1699	1468	866	866
query18	2317	413	404	404
query19	210	153	156	153
query20	82	81	81	81
query21	189	99	96	96
query22	5379	5189	5193	5189
query23	32410	32012	32046	32012
query24	7054	6515	6519	6515
query25	520	426	423	423
query26	531	165	162	162
query27	1912	291	295	291
query28	6132	2356	2329	2329
query29	2884	2794	2970	2794
query30	248	163	167	163
query31	921	731	757	731
query32	70	65	63	63
query33	400	263	250	250
query34	891	465	488	465
query35	1135	900	933	900
query36	1195	1221	1170	1170
query37	90	61	60	60
query38	3138	2950	2959	2950
query39	1376	1337	1338	1337
query40	210	100	94	94
query41	39	37	37	37
query42	87	83	81	81
query43	765	607	766	607
query44	1120	715	717	715
query45	246	232	230	230
query46	1220	965	971	965
query47	1775	1833	1706	1706
query48	1038	705	707	705
query49	630	372	375	372
query50	850	597	580	580
query51	4842	4701	4678	4678
query52	95	76	80	76
query53	441	323	319	319
query54	2665	2455	2466	2455
query55	91	71	75	71
query56	230	206	221	206
query57	1327	1155	1086	1086
query58	215	198	210	198
query59	4332	4068	3995	3995
query60	208	199	228	199
query61	98	96	100	96
query62	857	526	509	509
query63	487	342	336	336
query64	2416	1536	1333	1333
query65	3667	3559	3570	3559
query66	783	384	378	378
query67	15779	15403	15149	15149
query68	10387	650	644	644
query69	577	343	356	343
query70	2138	1277	1395	1277
query71	427	313	315	313
query72	6600	3503	3511	3503
query73	744	323	320	320
query74	6356	5924	5909	5909
query75	5390	3635	3744	3635
query76	6825	1150	1215	1150
query77	1188	277	259	259
query78	12411	11615	11637	11615
query79	8336	656	628	628
query80	1048	411	397	397
query81	455	240	233	233
query82	908	98	97	97
query83	176	143	140	140
query84	251	71	72	71
query85	883	326	322	322
query86	332	294	308	294
query87	3222	3013	3054	3013
query88	4253	2378	2364	2364
query89	487	306	305	305
query90	1949	212	212	212
query91	172	139	150	139
query92	57	57	54	54
query93	7141	587	596	587
query94	701	214	216	214
query95	1117	1055	1078	1055
query96	651	335	335	335
query97	6525	6284	6416	6284
query98	193	177	175	175
query99	3180	929	861	861
Total cold run time: 316792 ms
Total hot run time: 203822 ms

doris-robot avatar May 23 '24 09:05 doris-robot

ClickBench: Total hot run time: 30.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4fecbcb41efbe83863eac7ec83aa200cbed78388, data reload: false

query1	0.02	0.02	0.01
query2	0.08	0.02	0.02
query3	0.25	0.04	0.05
query4	1.80	0.07	0.06
query5	0.54	0.52	0.52
query6	1.28	0.61	0.62
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.51	0.50	0.48
query10	0.54	0.54	0.55
query11	0.13	0.08	0.09
query12	0.12	0.10	0.09
query13	0.62	0.62	0.61
query14	0.77	0.78	0.79
query15	0.78	0.75	0.76
query16	0.37	0.37	0.39
query17	1.01	1.00	1.02
query18	0.21	0.24	0.25
query19	1.94	1.87	1.85
query20	0.01	0.02	0.02
query21	15.46	0.58	0.55
query22	1.89	2.40	1.58
query23	17.39	1.03	0.98
query24	5.26	0.87	1.19
query25	0.40	0.10	0.05
query26	0.54	0.16	0.15
query27	0.04	0.05	0.04
query28	7.87	0.71	0.73
query29	12.61	2.27	2.26
query30	0.61	0.53	0.52
query31	2.82	0.40	0.37
query32	3.35	0.49	0.50
query33	3.09	3.14	3.07
query34	15.25	4.80	4.78
query35	4.86	4.84	4.83
query36	1.03	1.02	1.02
query37	0.06	0.05	0.04
query38	0.03	0.03	0.02
query39	0.03	0.01	0.01
query40	0.17	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.9 s
Total hot run time: 30.62 s

doris-robot avatar May 23 '24 09:05 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4fecbcb41efbe83863eac7ec83aa200cbed78388 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

doris-robot avatar May 23 '24 09:05 doris-robot

@SWJTU-ZhangLei @dataroaring Is it necessary to change in 2.0?

xiaokang avatar May 26 '24 08:05 xiaokang