doris icon indicating copy to clipboard operation
doris copied to clipboard

[feature](metrics)support be jvm metrics.

Open hubgeter opened this issue 9 months ago • 10 comments

Proposed changes

support be jvm metrics. if you curl http://be_host:webserver_port/metrics , you will get :

doris_be_jvm_heap_size_bytes{type="max"} 8589934592
doris_be_jvm_heap_size_bytes{type="committed"} 8589934592
doris_be_jvm_heap_size_bytes{type="used"} 364159504

doris_be_jvm_non_heap_size_bytes{type="committed"} 117899264
doris_be_jvm_non_heap_size_bytes{type="used"} 115330424

doris_be_jvm_young_size_bytes{type="used"} 255852544
doris_be_jvm_young_size_bytes{type="peak_used"} 255852544
doris_be_jvm_young_size_bytes{type="max"} 8589934592

doris_be_jvm_old_size_bytes{type="used"} 94393344
doris_be_jvm_old_size_bytes{type="peak_used"} 94393344
doris_be_jvm_old_size_bytes{type="max"} 8589934592

doris_be_jvm_gc{name="G1 Young Generation Count", type="count"} 3
doris_be_jvm_gc{name="G1 Young Generation Time", type="time"} 33
doris_be_jvm_gc{name="G1 Old Generation Count", type="count"} 0
doris_be_jvm_gc{name="G1 Old Generation Time", type="time"} 0

doris_be_jvm_thread{type="count"} 147
doris_be_jvm_thread{type="peak_count"} 147
doris_be_jvm_thread{type="new_count"} 0
doris_be_jvm_thread{type="runnable_count"} 25
doris_be_jvm_thread{type="blocked_count"} 0
doris_be_jvm_thread{type="waiting_count"} 48
doris_be_jvm_thread{type="timed_waiting_count"} 74
doris_be_jvm_thread{type="terminated_count"} 0

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

hubgeter avatar May 17 '24 11:05 hubgeter

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar May 17 '24 11:05 doris-robot

run buildall

hubgeter avatar May 18 '24 17:05 hubgeter

TeamCity be ut coverage result: Function Coverage: 35.69% (9006/25232) Line Coverage: 27.33% (74483/272527) Region Coverage: 26.57% (38520/144963) Branch Coverage: 23.41% (19661/83982) Coverage Report: http://coverage.selectdb-in.cc/coverage/4b39fed0721ccce5844a5af35a457c89e33f67e9_4b39fed0721ccce5844a5af35a457c89e33f67e9/report/index.html

doris-robot avatar May 18 '24 18:05 doris-robot

run p0

hubgeter avatar May 19 '24 16:05 hubgeter

run buildall

hubgeter avatar May 21 '24 08:05 hubgeter

TeamCity be ut coverage result: Function Coverage: 35.69% (9018/25267) Line Coverage: 27.31% (74538/272936) Region Coverage: 26.52% (38526/145278) Branch Coverage: 23.37% (19655/84094) Coverage Report: http://coverage.selectdb-in.cc/coverage/4b39fed0721ccce5844a5af35a457c89e33f67e9_4b39fed0721ccce5844a5af35a457c89e33f67e9/report/index.html

doris-robot avatar May 21 '24 09:05 doris-robot

TPC-DS: Total hot run time: 182986 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b39fed0721ccce5844a5af35a457c89e33f67e9, data reload: false

query1	924	381	369	369
query2	6245	2590	2329	2329
query3	6626	210	209	209
query4	23886	21918	21950	21918
query5	3521	425	426	425
query6	271	179	170	170
query7	4557	309	300	300
query8	244	190	191	190
query9	8498	2437	2445	2437
query10	434	246	266	246
query11	15388	14867	14790	14790
query12	116	87	83	83
query13	1584	385	381	381
query14	10326	8400	7522	7522
query15	249	171	166	166
query16	8202	255	250	250
query17	1783	545	534	534
query18	2084	273	267	267
query19	339	152	147	147
query20	88	85	80	80
query21	191	126	122	122
query22	5077	4874	4849	4849
query23	34101	33587	33486	33486
query24	10589	2946	2908	2908
query25	605	371	348	348
query26	1452	155	146	146
query27	2887	319	320	319
query28	7459	2042	2028	2028
query29	914	622	587	587
query30	250	171	172	171
query31	947	752	748	748
query32	96	49	50	49
query33	737	243	244	243
query34	1021	468	472	468
query35	808	672	678	672
query36	1031	945	894	894
query37	134	75	67	67
query38	2880	2777	2800	2777
query39	1621	1563	1579	1563
query40	243	123	124	123
query41	43	43	42	42
query42	99	92	93	92
query43	601	561	546	546
query44	1186	741	747	741
query45	254	259	252	252
query46	1105	700	703	700
query47	1958	1874	1900	1874
query48	386	302	301	301
query49	932	397	428	397
query50	759	377	385	377
query51	6789	6753	6771	6753
query52	100	85	90	85
query53	346	283	284	283
query54	871	423	428	423
query55	71	70	70	70
query56	241	219	226	219
query57	1211	1151	1152	1151
query58	214	195	195	195
query59	3434	3226	3213	3213
query60	258	230	232	230
query61	91	86	89	86
query62	675	471	478	471
query63	308	282	279	279
query64	9580	2237	1724	1724
query65	3732	3100	3142	3100
query66	1339	321	333	321
query67	15515	15071	14947	14947
query68	4484	542	536	536
query69	496	294	301	294
query70	1116	1172	1156	1156
query71	402	276	262	262
query72	7140	2536	2338	2338
query73	728	319	359	319
query74	6762	6343	6336	6336
query75	3400	2608	2589	2589
query76	2726	993	989	989
query77	369	258	258	258
query78	10772	10140	9993	9993
query79	2362	515	524	515
query80	963	424	422	422
query81	528	240	239	239
query82	643	91	92	91
query83	253	160	165	160
query84	243	83	88	83
query85	1869	268	265	265
query86	505	296	268	268
query87	3353	3128	3137	3128
query88	4199	2435	2441	2435
query89	465	379	376	376
query90	1976	192	185	185
query91	121	98	95	95
query92	62	47	47	47
query93	2285	506	503	503
query94	1148	179	176	176
query95	388	300	357	300
query96	598	269	272	269
query97	3167	3008	3042	3008
query98	235	212	214	212
query99	1098	886	887	886
Total cold run time: 285362 ms
Total hot run time: 182986 ms

doris-robot avatar May 21 '24 10:05 doris-robot

ClickBench: Total hot run time: 31.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b39fed0721ccce5844a5af35a457c89e33f67e9, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.08
query5	0.49	0.51	0.51
query6	1.12	0.71	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.04
query9	0.52	0.48	0.49
query10	0.54	0.55	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.79	0.76	0.78
query15	0.83	0.81	0.80
query16	0.37	0.37	0.38
query17	1.02	1.01	0.97
query18	0.22	0.22	0.27
query19	1.84	1.71	1.72
query20	0.01	0.00	0.01
query21	15.49	0.70	0.69
query22	4.30	6.53	2.72
query23	18.30	1.37	1.21
query24	1.32	0.27	0.29
query25	0.14	0.10	0.08
query26	0.25	0.16	0.16
query27	0.07	0.08	0.08
query28	13.51	1.01	1.00
query29	13.19	3.34	3.29
query30	0.24	0.05	0.05
query31	2.89	0.39	0.37
query32	3.29	0.47	0.47
query33	2.86	2.94	2.85
query34	17.09	4.37	4.44
query35	4.57	4.56	4.64
query36	0.65	0.47	0.45
query37	0.17	0.14	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.15
query41	0.09	0.05	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.62 s
Total hot run time: 31.32 s

doris-robot avatar May 21 '24 10:05 doris-robot

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar May 21 '24 15:05 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar May 21 '24 15:05 github-actions[bot]