doris
doris copied to clipboard
apache
[feature](compaction) Add an http action for visibility of compaction score on each tablet
Proposed changes
As title.
Usage:
curl http://be_ip:be_host/api/compaction_score?top_n=10Returns a json object contains compaction score for top n, n=top_n.
[
{
"compaction_score": "5",
"tablet_id": "42595"
},
{
"compaction_score": "5",
"tablet_id": "42587"
},
{
"compaction_score": "5",
"tablet_id": "42593"
},
{
"compaction_score": "5",
"tablet_id": "42597"
},
{
"compaction_score": "5",
"tablet_id": "42589"
},
{
"compaction_score": "5",
"tablet_id": "42599"
},
{
"compaction_score": "5",
"tablet_id": "42601"
},
{
"compaction_score": "5",
"tablet_id": "42591"
},
{
"compaction_score": "5",
"tablet_id": "42585"
},
{
"compaction_score": "4",
"tablet_id": "10034"
}
]
If top_n is not specified, return all compaction score for all tablets. If top_n is illegal, raise an error.
invalid argument: top_n=wrong
curl http://be_ip:be_host/api/compaction_score?sync_meta=truesync_metais only available on cloud mode, will sync meta from meta service. It can cooperate with top_n. If add paramsync_metaon non-cloud mode, will raise an error.
sync meta is only available for cloud mode
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TPC-H: Total hot run time: 41594 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a047be16ace337a026e965d759bd914628f531a0, data reload: false
------ Round 1 ----------------------------------
q1 18376 4237 4171 4171
q2 2763 220 212 212
q3 12142 1262 1361 1262
q4 10556 812 971 812
q5 7874 3023 2964 2964
q6 225 139 142 139
q7 1048 620 612 612
q8 9449 1857 1916 1857
q9 8613 6631 6622 6622
q10 8721 3837 3810 3810
q11 433 266 254 254
q12 412 228 228 228
q13 17752 2962 2959 2959
q14 274 243 244 243
q15 524 484 474 474
q16 485 400 385 385
q17 961 921 925 921
q18 8233 7425 7224 7224
q19 1388 1215 1215 1215
q20 575 311 335 311
q21 5325 4754 4640 4640
q22 347 281 279 279
Total cold run time: 116476 ms
Total hot run time: 41594 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4088 4052 4047 4047
q2 332 229 227 227
q3 2977 3022 2997 2997
q4 1896 1879 1871 1871
q5 5272 5259 5243 5243
q6 219 131 131 131
q7 2082 1716 1706 1706
q8 3245 3276 3287 3276
q9 8338 8289 8322 8289
q10 3730 3815 3834 3815
q11 540 462 455 455
q12 726 544 524 524
q13 14418 2958 2967 2958
q14 283 276 262 262
q15 514 475 482 475
q16 468 398 408 398
q17 1740 1711 1708 1708
q18 7765 7289 7288 7288
q19 1680 1666 1674 1666
q20 1998 1788 1770 1770
q21 5520 5261 5216 5216
q22 527 467 451 451
Total cold run time: 68358 ms
Total hot run time: 54773 ms
TPC-DS: Total hot run time: 169610 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a047be16ace337a026e965d759bd914628f531a0, data reload: false
query1 925 378 385 378
query2 6468 1665 1644 1644
query3 6679 213 225 213
query4 20333 17544 17318 17318
query5 4295 518 503 503
query6 287 172 162 162
query7 4600 287 311 287
query8 263 200 190 190
query9 8555 2379 2363 2363
query10 440 267 263 263
query11 10517 10064 10038 10038
query12 139 93 87 87
query13 1645 384 372 372
query14 9736 8577 7846 7846
query15 221 164 169 164
query16 7099 467 441 441
query17 967 560 573 560
query18 1927 297 290 290
query19 200 151 147 147
query20 97 87 88 87
query21 207 103 101 101
query22 4095 3908 4064 3908
query23 33855 33027 33163 33027
query24 10579 3105 3052 3052
query25 727 408 402 402
query26 1746 152 150 150
query27 2891 274 274 274
query28 6901 1956 1949 1949
query29 1362 430 432 430
query30 286 151 153 151
query31 961 763 764 763
query32 107 57 58 57
query33 715 333 320 320
query34 926 473 491 473
query35 857 730 736 730
query36 1004 864 857 857
query37 289 79 80 79
query38 2880 2772 2776 2772
query39 856 822 811 811
query40 290 118 115 115
query41 54 48 47 47
query42 129 103 106 103
query43 477 402 414 402
query44 1202 722 751 722
query45 208 180 186 180
query46 1076 806 796 796
query47 1815 1700 1708 1700
query48 368 298 293 293
query49 1218 432 525 432
query50 919 433 426 426
query51 6839 6672 6650 6650
query52 104 96 95 95
query53 251 178 178 178
query54 647 453 462 453
query55 77 73 76 73
query56 280 257 253 253
query57 1127 1026 1039 1026
query58 292 282 269 269
query59 2769 2306 2437 2306
query60 292 274 275 274
query61 99 97 96 96
query62 926 674 663 663
query63 223 184 187 184
query64 5873 1913 1899 1899
query65 3152 3109 3113 3109
query66 1462 338 362 338
query67 15572 14682 14719 14682
query68 7123 561 584 561
query69 680 378 315 315
query70 1130 1037 1087 1037
query71 519 277 268 268
query72 7630 2713 2521 2521
query73 978 323 359 323
query74 5990 5655 5622 5622
query75 3899 2723 2735 2723
query76 5418 1404 1414 1404
query77 677 305 306 305
query78 9512 8848 8848 8848
query79 3197 528 537 528
query80 1241 504 498 498
query81 562 223 223 223
query82 751 131 136 131
query83 208 167 171 167
query84 274 85 77 77
query85 1438 378 306 306
query86 430 281 298 281
query87 3277 3110 3107 3107
query88 3626 2402 2412 2402
query89 436 289 287 287
query90 2047 192 193 192
query91 128 107 103 103
query92 67 49 48 48
query93 3915 611 600 600
query94 952 294 262 262
query95 388 266 267 266
query96 611 279 284 279
query97 3216 3065 3031 3031
query98 223 195 191 191
query99 1664 1304 1281 1281
Total cold run time: 276585 ms
Total hot run time: 169610 ms
ClickBench: Total hot run time: 30.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a047be16ace337a026e965d759bd914628f531a0, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.03 0.04
query3 0.22 0.04 0.05
query4 1.68 0.07 0.07
query5 0.48 0.48 0.48
query6 1.13 0.71 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.05
query9 0.57 0.51 0.52
query10 0.57 0.58 0.57
query11 0.16 0.12 0.11
query12 0.15 0.12 0.13
query13 0.61 0.60 0.59
query14 0.78 0.80 0.80
query15 0.89 0.86 0.86
query16 0.35 0.35 0.36
query17 1.02 1.02 0.99
query18 0.21 0.21 0.22
query19 1.86 1.74 1.78
query20 0.01 0.02 0.01
query21 15.80 0.80 0.65
query22 3.59 7.53 1.63
query23 18.01 1.33 1.40
query24 2.26 0.23 0.22
query25 0.18 0.08 0.09
query26 0.31 0.22 0.21
query27 0.45 0.23 0.24
query28 13.18 0.99 0.97
query29 12.55 3.28 3.25
query30 0.26 0.05 0.05
query31 2.86 0.41 0.38
query32 3.27 0.48 0.48
query33 2.91 2.99 2.96
query34 15.44 4.27 4.21
query35 4.27 4.30 4.32
query36 0.67 0.47 0.49
query37 0.19 0.16 0.16
query38 0.17 0.15 0.16
query39 0.04 0.03 0.03
query40 0.16 0.13 0.13
query41 0.09 0.04 0.04
query42 0.06 0.05 0.04
query43 0.05 0.04 0.04
Total cold run time: 107.64 s
Total hot run time: 30.28 s