doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[feature])(function) add percentile_approx_weighted function
Proposed changes
https://github.com/apache/doris-website/pull/682
mysql [test]>select * from quantile_weighted_table order by k;
+------+------+
| k | w |
+------+------+
| 1 | 2 |
| 3 | 1 |
| 5 | 2 |
+------+------+
3 rows in set (0.04 sec)
mysql [test]>select percentile_approx_weighted(k,w,0.55) from quantile_weighted_table;
+----------------------------------------------------------------------------------------+
| percentile_approx_weighted(cast(k as DOUBLE), cast(w as DOUBLE), cast(0.55 as DOUBLE)) |
+----------------------------------------------------------------------------------------+
| 3.3333332538604736 |
+----------------------------------------------------------------------------------------+
1 row in set (0.05 sec)
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TeamCity be ut coverage result: Function Coverage: 35.70% (9017/25261) Line Coverage: 27.34% (74549/272672) Region Coverage: 26.55% (38537/145129) Branch Coverage: 23.40% (19662/84016) Coverage Report: http://coverage.selectdb-in.cc/coverage/e6db76fb2eb1075ddfc41ccf2480f404b6c62d78_e6db76fb2eb1075ddfc41ccf2480f404b6c62d78/report/index.html
TPC-DS: Total hot run time: 183358 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e6db76fb2eb1075ddfc41ccf2480f404b6c62d78, data reload: false
query1 919 379 368 368
query2 6292 2383 2398 2383
query3 6633 218 217 217
query4 23393 21916 21949 21916
query5 3567 410 422 410
query6 265 170 168 168
query7 4559 290 292 290
query8 231 193 188 188
query9 8464 2485 2460 2460
query10 421 253 248 248
query11 15240 14908 14886 14886
query12 123 94 85 85
query13 1570 382 376 376
query14 10957 7734 8449 7734
query15 246 186 166 166
query16 8183 262 263 262
query17 1775 569 569 569
query18 2093 271 263 263
query19 324 147 146 146
query20 87 81 85 81
query21 191 132 126 126
query22 5071 4888 4867 4867
query23 34494 33622 33597 33597
query24 11429 2938 2846 2846
query25 618 370 356 356
query26 1627 150 157 150
query27 2850 322 313 313
query28 7393 2006 2001 2001
query29 960 607 592 592
query30 266 176 171 171
query31 965 754 766 754
query32 90 50 49 49
query33 742 266 250 250
query34 992 477 463 463
query35 823 686 663 663
query36 1096 934 924 924
query37 168 71 71 71
query38 2950 2794 2813 2794
query39 1600 1576 1583 1576
query40 246 125 123 123
query41 47 42 43 42
query42 106 93 95 93
query43 610 562 550 550
query44 1206 724 735 724
query45 264 257 258 257
query46 1098 718 703 703
query47 1990 1915 1910 1910
query48 379 304 300 300
query49 975 393 382 382
query50 758 373 379 373
query51 6817 6910 6845 6845
query52 97 99 88 88
query53 356 283 297 283
query54 974 418 423 418
query55 77 72 73 72
query56 239 221 246 221
query57 1265 1175 1175 1175
query58 224 196 196 196
query59 3398 3087 3118 3087
query60 268 230 231 230
query61 88 87 88 87
query62 660 469 470 469
query63 308 281 282 281
query64 9697 2300 1722 1722
query65 3201 3109 3091 3091
query66 1354 337 355 337
query67 15560 15120 15065 15065
query68 6552 543 523 523
query69 527 302 308 302
query70 1120 1079 1152 1079
query71 473 262 273 262
query72 7580 2527 2356 2356
query73 742 316 313 313
query74 6709 6247 6205 6205
query75 4140 2600 2610 2600
query76 4244 946 980 946
query77 665 265 260 260
query78 10640 10298 10110 10110
query79 2792 515 520 515
query80 1337 468 427 427
query81 493 246 240 240
query82 1148 93 94 93
query83 192 170 176 170
query84 263 87 84 84
query85 1385 269 256 256
query86 456 310 331 310
query87 3333 3105 3121 3105
query88 3848 2325 2311 2311
query89 470 386 378 378
query90 2020 182 180 180
query91 121 96 96 96
query92 60 47 49 47
query93 2339 483 480 480
query94 1161 176 174 174
query95 385 299 292 292
query96 602 317 258 258
query97 3169 3038 2979 2979
query98 237 212 201 201
query99 1336 911 903 903
Total cold run time: 292468 ms
Total hot run time: 183358 ms
ClickBench: Total hot run time: 30.79 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e6db76fb2eb1075ddfc41ccf2480f404b6c62d78, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.22 0.06 0.06
query4 1.67 0.08 0.08
query5 0.50 0.49 0.51
query6 1.13 0.71 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.50 0.49
query10 0.53 0.54 0.53
query11 0.16 0.12 0.11
query12 0.14 0.12 0.13
query13 0.61 0.58 0.59
query14 0.79 0.79 0.78
query15 0.84 0.80 0.81
query16 0.35 0.36 0.37
query17 0.98 1.02 0.95
query18 0.21 0.23 0.28
query19 1.90 1.83 1.80
query20 0.02 0.01 0.01
query21 15.47 0.69 0.68
query22 4.37 7.04 2.19
query23 18.28 1.39 1.22
query24 2.10 0.21 0.20
query25 0.13 0.09 0.08
query26 0.28 0.17 0.17
query27 0.07 0.07 0.08
query28 13.29 1.02 1.00
query29 13.20 3.31 3.25
query30 0.24 0.06 0.05
query31 2.87 0.40 0.39
query32 3.27 0.46 0.47
query33 2.86 2.93 2.92
query34 16.95 4.40 4.42
query35 4.56 4.47 4.58
query36 0.67 0.48 0.45
query37 0.17 0.16 0.15
query38 0.15 0.14 0.14
query39 0.04 0.04 0.03
query40 0.16 0.14 0.15
query41 0.09 0.04 0.05
query42 0.05 0.04 0.04
query43 0.04 0.03 0.04
Total cold run time: 110.1 s
Total hot run time: 30.79 s
TeamCity be ut coverage result: Function Coverage: 35.69% (9018/25265) Line Coverage: 27.34% (74587/272787) Region Coverage: 26.55% (38539/145173) Branch Coverage: 23.40% (19666/84040) Coverage Report: http://coverage.selectdb-in.cc/coverage/6ef7bb738f8de338b7bb0e98afc5d8e5644f2524_6ef7bb738f8de338b7bb0e98afc5d8e5644f2524/report/index.html
TPC-H: Total hot run time: 40119 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ef7bb738f8de338b7bb0e98afc5d8e5644f2524, data reload: false
------ Round 1 ----------------------------------
q1 17919 4463 4364 4364
q2 2582 197 195 195
q3 11109 1180 1180 1180
q4 10527 753 735 735
q5 8427 2746 2656 2656
q6 229 137 139 137
q7 1044 626 590 590
q8 9422 2145 2070 2070
q9 9149 6542 6521 6521
q10 8949 3718 3718 3718
q11 445 250 238 238
q12 553 220 216 216
q13 17768 3040 2981 2981
q14 257 221 228 221
q15 509 470 470 470
q16 480 393 380 380
q17 948 759 726 726
q18 8226 7541 7454 7454
q19 4538 1511 1537 1511
q20 649 309 311 309
q21 5136 3171 4034 3171
q22 348 276 281 276
Total cold run time: 119214 ms
Total hot run time: 40119 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4363 4221 4225 4221
q2 383 261 266 261
q3 3028 2795 2794 2794
q4 1889 1644 1580 1580
q5 5305 5278 5311 5278
q6 212 123 128 123
q7 2255 1826 1900 1826
q8 3208 3357 3381 3357
q9 8385 8359 8380 8359
q10 3907 3707 3720 3707
q11 587 492 483 483
q12 762 591 590 590
q13 16564 2987 2986 2986
q14 293 285 258 258
q15 521 471 479 471
q16 467 425 411 411
q17 1778 1510 1480 1480
q18 7731 7528 7493 7493
q19 1655 1539 1534 1534
q20 1988 1786 1807 1786
q21 5185 4857 4960 4857
q22 585 500 504 500
Total cold run time: 71051 ms
Total hot run time: 54355 ms
TPC-DS: Total hot run time: 181390 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ef7bb738f8de338b7bb0e98afc5d8e5644f2524, data reload: false
query1 917 384 370 370
query2 6448 2506 2331 2331
query3 6651 214 213 213
query4 23368 21267 21183 21183
query5 4169 428 423 423
query6 262 173 171 171
query7 4590 290 284 284
query8 234 186 196 186
query9 8534 2391 2374 2374
query10 442 250 249 249
query11 14756 14114 14118 14114
query12 131 89 87 87
query13 1646 367 377 367
query14 10852 7880 7791 7791
query15 215 180 179 179
query16 7908 266 258 258
query17 1860 585 586 585
query18 1978 273 266 266
query19 209 154 150 150
query20 90 90 83 83
query21 191 142 134 134
query22 5070 4802 4851 4802
query23 34201 33673 33152 33152
query24 11896 2950 2876 2876
query25 682 364 367 364
query26 1783 155 152 152
query27 2998 320 332 320
query28 7641 2022 2019 2019
query29 1078 606 608 606
query30 312 181 171 171
query31 998 769 745 745
query32 94 52 58 52
query33 733 268 247 247
query34 1026 486 479 479
query35 858 694 679 679
query36 1070 921 957 921
query37 148 69 70 69
query38 2917 2818 2727 2727
query39 1615 1681 1568 1568
query40 272 128 126 126
query41 49 43 43 43
query42 106 97 97 97
query43 583 538 533 533
query44 1273 728 735 728
query45 260 254 258 254
query46 1094 712 721 712
query47 2017 1928 1895 1895
query48 375 302 291 291
query49 1158 385 414 385
query50 764 392 378 378
query51 6967 6846 6802 6802
query52 104 85 91 85
query53 351 281 284 281
query54 830 422 433 422
query55 74 75 72 72
query56 244 219 231 219
query57 1241 1172 1181 1172
query58 233 201 201 201
query59 3433 3189 3330 3189
query60 254 231 241 231
query61 92 86 85 85
query62 662 485 467 467
query63 309 281 284 281
query64 9804 2203 1724 1724
query65 3158 3107 3086 3086
query66 1388 339 383 339
query67 15555 14952 15005 14952
query68 6095 524 535 524
query69 539 308 308 308
query70 1161 1134 1159 1134
query71 472 266 269 266
query72 7752 2595 2344 2344
query73 756 318 319 318
query74 6692 6161 6227 6161
query75 3978 2643 2640 2640
query76 3738 994 993 993
query77 630 264 272 264
query78 10633 10021 10018 10018
query79 2579 516 514 514
query80 1307 474 433 433
query81 510 247 241 241
query82 762 95 101 95
query83 202 171 171 171
query84 277 86 88 86
query85 1405 270 265 265
query86 402 312 296 296
query87 3310 3158 3130 3130
query88 4316 2329 2340 2329
query89 481 386 394 386
query90 1999 185 189 185
query91 123 99 99 99
query92 60 46 47 46
query93 3707 503 505 503
query94 1155 177 188 177
query95 391 303 301 301
query96 607 264 262 262
query97 3206 3009 3041 3009
query98 242 228 211 211
query99 1264 884 911 884
Total cold run time: 293972 ms
Total hot run time: 181390 ms
ClickBench: Total hot run time: 30.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ef7bb738f8de338b7bb0e98afc5d8e5644f2524, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.69 0.07 0.07
query5 0.49 0.48 0.51
query6 1.13 0.72 0.72
query7 0.02 0.02 0.02
query8 0.06 0.04 0.04
query9 0.53 0.47 0.49
query10 0.54 0.55 0.54
query11 0.16 0.12 0.11
query12 0.14 0.12 0.11
query13 0.59 0.59 0.60
query14 0.80 0.77 0.79
query15 0.83 0.80 0.82
query16 0.38 0.36 0.37
query17 1.02 0.95 1.03
query18 0.25 0.25 0.23
query19 1.78 1.79 1.75
query20 0.01 0.01 0.01
query21 15.71 0.65 0.64
query22 4.54 7.75 1.66
query23 18.29 1.49 1.30
query24 2.00 0.22 0.21
query25 0.15 0.08 0.09
query26 0.27 0.18 0.17
query27 0.08 0.08 0.08
query28 13.39 1.09 1.04
query29 12.68 3.23 3.22
query30 0.25 0.06 0.06
query31 2.88 0.39 0.38
query32 3.31 0.47 0.46
query33 2.84 2.91 2.93
query34 17.03 4.44 4.54
query35 4.48 4.53 4.45
query36 0.65 0.46 0.46
query37 0.18 0.16 0.16
query38 0.15 0.15 0.14
query39 0.05 0.04 0.04
query40 0.18 0.14 0.14
query41 0.09 0.05 0.05
query42 0.05 0.05 0.04
query43 0.04 0.03 0.04
Total cold run time: 110.06 s
Total hot run time: 30.3 s
TeamCity be ut coverage result: Function Coverage: 35.69% (9018/25265) Line Coverage: 27.34% (74573/272787) Region Coverage: 26.55% (38541/145173) Branch Coverage: 23.40% (19666/84040) Coverage Report: http://coverage.selectdb-in.cc/coverage/6ef7bb738f8de338b7bb0e98afc5d8e5644f2524_6ef7bb738f8de338b7bb0e98afc5d8e5644f2524/report/index.html
TPC-DS: Total hot run time: 183104 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ef7bb738f8de338b7bb0e98afc5d8e5644f2524, data reload: false
query1 904 394 365 365
query2 6292 2329 2316 2316
query3 6636 218 219 218
query4 22802 21835 22076 21835
query5 3537 442 432 432
query6 279 180 169 169
query7 4588 301 302 301
query8 247 184 186 184
query9 8690 2441 2413 2413
query10 407 246 246 246
query11 15087 14877 14719 14719
query12 123 91 85 85
query13 1593 373 362 362
query14 10237 7844 8521 7844
query15 241 178 162 162
query16 8145 264 259 259
query17 1707 549 537 537
query18 2104 273 258 258
query19 191 150 141 141
query20 89 81 86 81
query21 190 133 124 124
query22 5104 4887 4864 4864
query23 34077 34067 33679 33679
query24 10738 2920 2919 2919
query25 601 368 366 366
query26 1126 153 151 151
query27 2431 316 311 311
query28 7186 1999 2006 1999
query29 849 629 591 591
query30 258 169 177 169
query31 951 737 728 728
query32 93 50 52 50
query33 727 235 244 235
query34 1054 464 466 464
query35 792 671 663 663
query36 1097 905 917 905
query37 131 67 70 67
query38 2943 2785 2747 2747
query39 1606 1536 1587 1536
query40 204 128 126 126
query41 49 42 41 41
query42 101 94 95 94
query43 565 560 579 560
query44 1204 725 735 725
query45 264 253 248 248
query46 1074 734 710 710
query47 1976 1905 1885 1885
query48 366 300 289 289
query49 894 392 404 392
query50 742 373 371 371
query51 6932 6854 6726 6726
query52 103 88 92 88
query53 351 289 287 287
query54 912 458 415 415
query55 74 72 72 72
query56 244 223 220 220
query57 1238 1130 1156 1130
query58 224 196 195 195
query59 3438 3137 3168 3137
query60 258 226 235 226
query61 87 88 99 88
query62 654 458 472 458
query63 317 283 277 277
query64 8813 2238 1717 1717
query65 3159 3149 3087 3087
query66 810 333 335 333
query67 15486 15068 15024 15024
query68 4545 520 525 520
query69 502 302 298 298
query70 1219 1101 1174 1101
query71 427 263 267 263
query72 7431 2545 2390 2390
query73 708 317 317 317
query74 6609 6203 6230 6203
query75 3635 2608 2608 2608
query76 3016 965 1012 965
query77 613 265 259 259
query78 10564 10191 10394 10191
query79 2078 509 514 509
query80 966 432 423 423
query81 527 245 240 240
query82 627 94 96 94
query83 255 165 183 165
query84 239 84 81 81
query85 1733 276 261 261
query86 481 300 301 300
query87 3307 3151 3137 3137
query88 4256 2336 2339 2336
query89 471 382 376 376
query90 1973 187 182 182
query91 122 95 98 95
query92 61 46 48 46
query93 2286 477 493 477
query94 1162 181 179 179
query95 378 294 292 292
query96 592 281 262 262
query97 3145 3025 3029 3025
query98 246 218 214 214
query99 1205 919 879 879
Total cold run time: 281771 ms
Total hot run time: 183104 ms
ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ef7bb738f8de338b7bb0e98afc5d8e5644f2524, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.23 0.06 0.05
query4 1.68 0.09 0.09
query5 0.47 0.50 0.51
query6 1.13 0.72 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.53 0.49 0.49
query10 0.55 0.55 0.54
query11 0.16 0.11 0.12
query12 0.14 0.11 0.12
query13 0.60 0.58 0.60
query14 0.78 0.79 0.77
query15 0.83 0.81 0.80
query16 0.37 0.36 0.36
query17 1.01 1.02 1.03
query18 0.23 0.22 0.26
query19 1.81 1.80 1.79
query20 0.01 0.00 0.01
query21 15.57 0.69 0.66
query22 4.22 6.63 2.42
query23 18.31 1.35 1.23
query24 1.37 0.39 0.22
query25 0.15 0.08 0.08
query26 0.27 0.17 0.17
query27 0.08 0.07 0.08
query28 13.36 1.02 1.00
query29 13.31 3.22 3.27
query30 0.24 0.06 0.05
query31 2.87 0.40 0.38
query32 3.28 0.47 0.46
query33 2.82 2.93 2.83
query34 17.15 4.42 4.39
query35 4.52 4.50 4.67
query36 0.65 0.46 0.46
query37 0.17 0.16 0.16
query38 0.15 0.15 0.14
query39 0.05 0.03 0.04
query40 0.15 0.14 0.15
query41 0.09 0.05 0.05
query42 0.05 0.04 0.04
query43 0.03 0.04 0.03
Total cold run time: 109.58 s
Total hot run time: 31 s
PR approved by at least one committer and no changes requested.