doris
doris copied to clipboard
apache
[feature](agg-func) linear_histogram
Proposed changes
add aggregate function: linear_histogram
LINEAR_HISTOGRAM
Description
linear_histogram(expr, DOUBLE interval[, DOUBLE offset)
The linear_histogram function is used to describe the distribution of the data, It uses an "equal width" bucking strategy, and divides the data into buckets according to the value of the data.
Parameter description:
- interval: Required. The bucket width.
- offset: Optional. Default is 0. It should be in [0, interval).
Example
mysql> select linear_histogram(a, 2) from histogram_test;
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| linear_histogram(a, cast(2 as DOUBLE)) |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"num_buckets":6,"buckets":[{"lower":0.0,"upper":2.0,"count":2,"acc_count":2},{"lower":2.0,"upper":4.0,"count":4,"acc_count":6},{"lower":4.0,"upper":6.0,"count":4,"acc_count":10},{"lower":6.0,"upper":8.0,"count":4,"acc_count":14},{"lower":8.0,"upper":10.0,"count":4,"acc_count":18},{"lower":10.0,"upper":12.0,"count":2,"acc_count":20}]} |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.07 sec)
mysql> select linear_histogram(a, 2, 1) from histogram_test;
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| linear_histogram(a, cast(2 as DOUBLE), cast(1 as DOUBLE)) |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"num_buckets":5,"buckets":[{"lower":1.0,"upper":3.0,"count":4,"acc_count":4},{"lower":3.0,"upper":5.0,"count":4,"acc_count":8},{"lower":5.0,"upper":7.0,"count":4,"acc_count":12},{"lower":7.0,"upper":9.0,"count":4,"acc_count":16},{"lower":9.0,"upper":11.0,"count":4,"acc_count":20}]} |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.04 sec)
Field description:
- num_buckets: The number of buckets.
- buckets: All buckets.
- lower: Lower bound of the bucket. (included)
- upper: Upper bound of the bucket. (not included)
- count: The number of elements contained in the bucket.
- acc_count: Accumulated count.
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TPC-H: Total hot run time: 38361 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 721ba15860fea6304ab52a0d162fb9b1a95f6ab6, data reload: false
------ Round 1 ----------------------------------
q1 18134 4448 4390 4390
q2 2295 208 210 208
q3 10425 1137 1103 1103
q4 10186 748 854 748
q5 7760 2842 2768 2768
q6 281 160 161 160
q7 1020 654 646 646
q8 9389 2107 2077 2077
q9 7300 6560 6492 6492
q10 7033 2199 2290 2199
q11 522 279 280 279
q12 431 268 267 267
q13 17971 3000 3009 3000
q14 300 258 260 258
q15 576 522 523 522
q16 512 416 405 405
q17 982 731 727 727
q18 7436 6967 6675 6675
q19 7254 1098 1041 1041
q20 741 356 349 349
q21 3923 2994 3064 2994
q22 1153 1053 1053 1053
Total cold run time: 115624 ms
Total hot run time: 38361 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4472 4309 4291 4291
q2 391 314 313 313
q3 2850 2664 2590 2590
q4 1901 1629 1645 1629
q5 5380 5405 5363 5363
q6 237 145 145 145
q7 2070 1709 1771 1709
q8 3198 3353 3295 3295
q9 8486 8373 8394 8373
q10 3405 3165 3154 3154
q11 611 522 524 522
q12 826 635 652 635
q13 17517 3029 3015 3015
q14 345 310 321 310
q15 553 529 527 527
q16 510 453 442 442
q17 1789 1543 1500 1500
q18 7661 7587 7447 7447
q19 1744 1682 1644 1644
q20 2021 1822 1843 1822
q21 5323 5226 5096 5096
q22 1172 1062 1033 1033
Total cold run time: 72462 ms
Total hot run time: 54855 ms
TPC-DS: Total hot run time: 190978 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 721ba15860fea6304ab52a0d162fb9b1a95f6ab6, data reload: false
query1 991 394 398 394
query2 6751 1977 1880 1880
query3 6666 228 240 228
query4 29616 23308 23167 23167
query5 4417 722 699 699
query6 314 213 208 208
query7 4621 338 322 322
query8 480 441 430 430
query9 8611 2613 2577 2577
query10 510 361 346 346
query11 17879 15012 15299 15012
query12 183 142 139 139
query13 1716 451 439 439
query14 10039 6840 6745 6745
query15 289 204 192 192
query16 8016 514 497 497
query17 1616 592 578 578
query18 2149 348 357 348
query19 253 171 168 168
query20 142 140 141 140
query21 244 145 142 142
query22 4312 4031 4179 4031
query23 34037 33457 33380 33380
query24 11275 2891 2900 2891
query25 664 417 435 417
query26 1166 179 183 179
query27 2765 303 307 303
query28 7512 2165 2126 2126
query29 843 459 441 441
query30 334 190 183 183
query31 1050 853 805 805
query32 122 83 78 78
query33 808 345 343 343
query34 911 513 504 504
query35 880 752 755 752
query36 1115 958 916 916
query37 178 105 106 105
query38 4083 3907 3844 3844
query39 1548 1544 1468 1468
query40 243 159 155 155
query41 145 142 142 142
query42 140 118 123 118
query43 541 511 503 503
query44 1280 787 797 787
query45 227 199 229 199
query46 1127 785 767 767
query47 1933 1877 1875 1875
query48 412 342 344 342
query49 1263 585 580 580
query50 868 476 463 463
query51 6878 6765 6830 6765
query52 120 110 111 110
query53 299 232 226 226
query54 925 495 495 495
query55 92 90 89 89
query56 326 310 311 310
query57 1236 1131 1118 1118
query58 312 301 316 301
query59 2942 2710 2754 2710
query60 351 329 332 329
query61 151 147 147 147
query62 895 692 688 688
query63 257 229 227 227
query64 5341 2397 1894 1894
query65 3254 3158 3173 3158
query66 1156 691 728 691
query67 15351 14808 15069 14808
query68 6434 580 582 580
query69 726 413 334 334
query70 1239 1183 1151 1151
query71 577 315 324 315
query72 7677 2326 2073 2073
query73 846 353 359 353
query74 9231 8830 8880 8830
query75 4985 2731 2739 2731
query76 4725 1027 1023 1023
query77 896 442 499 442
query78 11269 9693 9876 9693
query79 13109 568 558 558
query80 1028 605 617 605
query81 610 258 252 252
query82 598 159 167 159
query83 416 214 213 213
query84 288 97 96 96
query85 828 358 348 348
query86 461 336 302 302
query87 4475 4440 4315 4315
query88 4998 2456 2646 2456
query89 511 332 330 330
query90 2109 229 231 229
query91 156 131 128 128
query92 89 112 82 82
query93 5833 568 548 548
query94 1040 313 323 313
query95 388 289 294 289
query96 636 290 280 280
query97 3198 3046 3017 3017
query98 250 226 240 226
query99 1517 1334 1293 1293
Total cold run time: 318314 ms
Total hot run time: 190978 ms
ClickBench: Total hot run time: 30.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 721ba15860fea6304ab52a0d162fb9b1a95f6ab6, data reload: false
query1 0.04 0.05 0.04
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.08
query5 0.50 0.49 0.49
query6 1.12 0.73 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.56 0.50 0.48
query10 0.55 0.55 0.53
query11 0.15 0.12 0.12
query12 0.16 0.12 0.13
query13 0.63 0.62 0.60
query14 0.77 0.78 0.79
query15 0.85 0.82 0.84
query16 0.38 0.36 0.36
query17 1.02 1.04 0.97
query18 0.22 0.23 0.22
query19 1.79 1.67 1.69
query20 0.01 0.01 0.01
query21 15.41 0.69 0.67
query22 4.09 7.93 1.62
query23 18.30 1.38 1.31
query24 2.04 0.25 0.23
query25 0.16 0.09 0.08
query26 0.32 0.24 0.24
query27 0.45 0.23 0.23
query28 13.26 1.03 1.00
query29 12.62 3.35 3.32
query30 0.43 0.26 0.24
query31 2.82 0.41 0.40
query32 3.23 0.49 0.50
query33 2.94 2.96 2.96
query34 17.10 4.39 4.35
query35 4.48 4.46 4.47
query36 0.68 0.49 0.50
query37 0.22 0.18 0.17
query38 0.18 0.17 0.17
query39 0.07 0.06 0.06
query40 0.18 0.14 0.15
query41 0.12 0.07 0.07
query42 0.08 0.08 0.06
query43 0.07 0.05 0.06
Total cold run time: 110.05 s
Total hot run time: 30.97 s
@superdiaodiao @yiguolei Hello, is this class GenerateFunction used to generate new functions? When I run the test function generate(), I got some errors:
java.lang.IllegalStateException: Unsupported generate code by data type: decimalv3(18,0)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.getDataTypeAndInstance(GenerateFunction.java:1056)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.generateSignatures(GenerateFunction.java:683)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.generateFunctions(GenerateFunction.java:596)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.lambda$collectFunctionCodes$9(GenerateFunction.java:549)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.collectFunctionCodes(GenerateFunction.java:556)
at org.apache.doris.nereids.rules.analysis.GenerateFunction.generate(GenerateFunction.java:276)
Am I using it incorrectly?
@superdiaodiao @yiguolei Hello, is this ...... Am I using it incorrectly?
Actually, we have to add FE codes manually rather than generating it, since generating code was out of date and served as a temporary method before.
@superdiaodiao @yiguolei Hello, is this ...... Am I using it incorrectly?
Actually, we have to add FE codes manually rather than generating it, since generating code was out of date and served as a temporary method before.
Thank you. I add it manually. But when I did the e2e test, I got a strange error. (This error did not occur when I did unit test in the be)
The agg function linear_histogram returns a json str.
I got different double values in json with same query and data.
The test case:
CREATE TABLE IF NOT EXISTS histogram_test
(
a int
)
ENGINE = OLAP
DUPLICATE KEY(a)
DISTRIBUTED BY HASH(a) BUCKETS AUTO
PROPERTIES (
"replication_num" = "1"
)
;
insert into histogram_test values (1);
insert into histogram_test values (2);
insert into histogram_test values (3);
insert into histogram_test values (4);
insert into histogram_test values (5);
insert into histogram_test values (6);
insert into histogram_test values (7);
insert into histogram_test values (8);
select linear_histogram(a, 2) from histogram_test;
The result of the first execution:
{"num_buckets":5,"buckets":[{"lower":4.6737435351163e-310,"upper":9.34748707015187e-310,"count":1,"acc_count":1},{"lower":9.34748707015187e-310,"upper":1.402123060518747e-309,"count":2,"acc_count":3},{"lower":1.402123060518747e-309,"upper":1.869497414022306e-309,"count":2,"acc_count":5},{"lower":1.869497414022306e-309,"upper":2.336871767525866e-309,"count":2,"acc_count":7},{"lower":2.336871767525866e-309,"upper":2.804246121029425e-309,"count":1,"acc_count":8}]}
The result of the second execution:
{"num_buckets":5,"buckets":[{"lower":9.399408234378048e218,"upper":3.8141098555780214e228,"count":1,"acc_count":1},{"lower":3.8141098555780214e228,"upper":7.628219710216102e228,"count":2,"acc_count":3},{"lower":7.628219710216102e228,"upper":1.1442329564854183e229,"count":2,"acc_count":5},{"lower":1.1442329564854183e229,"upper":1.5256439419492263e229,"count":2,"acc_count":7},{"lower":1.5256439419492263e229,"upper":1.9070549274130343e229,"count":1,"acc_count":8}]}
The result of the third execution:
{"num_buckets":5,"buckets":[{"lower":0.0,"upper":2.0,"count":1,"acc_count":1},{"lower":2.0,"upper":4.0,"count":2,"acc_count":3},{"lower":4.0,"upper":6.0,"count":2,"acc_count":5},{"lower":6.0,"upper":8.0,"count":2,"acc_count":7},{"lower":8.0,"upper":10.0,"count":1,"acc_count":8}]}
...
Only the result of the third execution is correct. I am very confused about this. Why this happens?
@superdiaodiao @yiguolei Hello, is this ...... Am I using it incorrectly?
Actually, we have to add FE codes manually rather than generating it, since generating code was out of date and served as a temporary method before.
Thank you. I add it manually. But when I did the e2e test, I got a strange error. (This error did not occur when I did unit test in the be) The agg function
linear_histogramreturns a json str. I got different double values in json with same query and data. The test case:CREATE TABLE IF NOT EXISTS histogram_test ( a int ) ENGINE = OLAP DUPLICATE KEY(a) DISTRIBUTED BY HASH(a) BUCKETS AUTO PROPERTIES ( "replication_num" = "1" ) ; insert into histogram_test values (1); insert into histogram_test values (2); insert into histogram_test values (3); insert into histogram_test values (4); insert into histogram_test values (5); insert into histogram_test values (6); insert into histogram_test values (7); insert into histogram_test values (8); select linear_histogram(a, 2) from histogram_test;The result of the first execution:
{"num_buckets":5,"buckets":[{"lower":4.6737435351163e-310,"upper":9.34748707015187e-310,"count":1,"acc_count":1},{"lower":9.34748707015187e-310,"upper":1.402123060518747e-309,"count":2,"acc_count":3},{"lower":1.402123060518747e-309,"upper":1.869497414022306e-309,"count":2,"acc_count":5},{"lower":1.869497414022306e-309,"upper":2.336871767525866e-309,"count":2,"acc_count":7},{"lower":2.336871767525866e-309,"upper":2.804246121029425e-309,"count":1,"acc_count":8}]}The result of the second execution:
{"num_buckets":5,"buckets":[{"lower":9.399408234378048e218,"upper":3.8141098555780214e228,"count":1,"acc_count":1},{"lower":3.8141098555780214e228,"upper":7.628219710216102e228,"count":2,"acc_count":3},{"lower":7.628219710216102e228,"upper":1.1442329564854183e229,"count":2,"acc_count":5},{"lower":1.1442329564854183e229,"upper":1.5256439419492263e229,"count":2,"acc_count":7},{"lower":1.5256439419492263e229,"upper":1.9070549274130343e229,"count":1,"acc_count":8}]}The result of the third execution:
{"num_buckets":5,"buckets":[{"lower":0.0,"upper":2.0,"count":1,"acc_count":1},{"lower":2.0,"upper":4.0,"count":2,"acc_count":3},{"lower":4.0,"upper":6.0,"count":2,"acc_count":5},{"lower":6.0,"upper":8.0,"count":2,"acc_count":7},{"lower":8.0,"upper":10.0,"count":1,"acc_count":8}]}...
Only the result of the third execution is correct. I am very confused about this. Why this happens?
Sorry, I have fixed it. I just forgot to set parameters in merge function~~
TPC-H: Total hot run time: 37898 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b2f87cd3b48345dacc417d984aaa716aff3cf40, data reload: false
------ Round 1 ----------------------------------
q1 18151 4461 4383 4383
q2 3112 181 187 181
q3 11535 1142 1164 1142
q4 10350 687 755 687
q5 7792 2898 2884 2884
q6 229 136 138 136
q7 967 614 602 602
q8 9326 2041 2035 2035
q9 7298 6550 6559 6550
q10 6992 2214 2219 2214
q11 466 237 240 237
q12 394 220 216 216
q13 17797 3058 3001 3001
q14 287 233 240 233
q15 527 485 488 485
q16 479 389 386 386
q17 981 682 737 682
q18 7326 6832 6804 6804
q19 1396 1000 1075 1000
q20 669 339 344 339
q21 4010 3100 2691 2691
q22 1134 1036 1010 1010
Total cold run time: 111218 ms
Total hot run time: 37898 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4320 4268 4269 4268
q2 378 270 272 270
q3 2901 2649 2649 2649
q4 1961 1647 1644 1644
q5 5389 5404 5397 5397
q6 218 129 127 127
q7 2123 1685 1690 1685
q8 3206 3341 3315 3315
q9 8382 8398 8385 8385
q10 3440 3210 3167 3167
q11 593 505 493 493
q12 784 578 616 578
q13 11980 3039 2999 2999
q14 312 278 280 278
q15 523 472 482 472
q16 480 430 417 417
q17 1774 1487 1540 1487
q18 7616 7299 7622 7299
q19 1663 1557 1506 1506
q20 2066 1829 1794 1794
q21 5482 5195 5187 5187
q22 1145 1108 1048 1048
Total cold run time: 66736 ms
Total hot run time: 54465 ms
TPC-DS: Total hot run time: 186595 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b2f87cd3b48345dacc417d984aaa716aff3cf40, data reload: false
query1 902 376 364 364
query2 6449 2029 1826 1826
query3 6645 204 218 204
query4 26604 23212 23102 23102
query5 4148 497 482 482
query6 257 161 159 159
query7 4588 300 288 288
query8 265 211 199 199
query9 8566 2452 2459 2452
query10 436 285 258 258
query11 16761 14892 14860 14860
query12 151 101 99 99
query13 1619 393 371 371
query14 9795 7068 6921 6921
query15 248 171 174 171
query16 7944 459 482 459
query17 1579 580 591 580
query18 2032 301 311 301
query19 190 145 146 145
query20 117 112 110 110
query21 210 104 102 102
query22 4502 4296 4125 4125
query23 34071 33424 33485 33424
query24 11078 2825 2913 2825
query25 614 379 391 379
query26 1162 159 161 159
query27 2858 283 283 283
query28 7616 2023 2023 2023
query29 816 422 413 413
query30 305 157 151 151
query31 967 768 741 741
query32 98 59 60 59
query33 757 290 291 290
query34 975 491 503 491
query35 866 723 716 716
query36 1081 941 939 939
query37 159 85 80 80
query38 4032 3899 3845 3845
query39 1431 1385 1389 1385
query40 205 118 118 118
query41 47 48 45 45
query42 114 102 96 96
query43 506 472 457 457
query44 1212 742 754 742
query45 199 167 171 167
query46 1106 742 752 742
query47 1879 1777 1805 1777
query48 362 311 298 298
query49 1088 425 437 425
query50 819 415 430 415
query51 7311 7097 7155 7097
query52 99 88 89 88
query53 253 180 181 180
query54 937 457 452 452
query55 78 81 76 76
query56 281 266 260 260
query57 1173 1093 1081 1081
query58 237 234 249 234
query59 2922 2895 2766 2766
query60 296 266 275 266
query61 116 99 97 97
query62 823 664 666 664
query63 219 181 183 181
query64 6437 2330 1768 1768
query65 3202 3112 3148 3112
query66 1365 344 349 344
query67 15593 15230 15278 15230
query68 4044 568 562 562
query69 409 275 277 275
query70 1108 1080 1114 1080
query71 337 287 278 278
query72 6418 2331 2068 2068
query73 749 321 334 321
query74 9022 8867 8741 8741
query75 3377 2666 2673 2666
query76 2276 1036 930 930
query77 483 315 329 315
query78 11047 9451 9080 9080
query79 1023 538 523 523
query80 739 514 519 514
query81 561 228 229 228
query82 239 146 147 146
query83 223 151 157 151
query84 229 76 79 76
query85 733 294 342 294
query86 303 286 297 286
query87 4436 4171 4215 4171
query88 3034 2356 2362 2356
query89 380 294 282 282
query90 1985 201 208 201
query91 126 102 100 100
query92 63 53 53 53
query93 1049 549 538 538
query94 884 304 315 304
query95 358 268 269 268
query96 600 273 273 273
query97 3211 3077 3129 3077
query98 210 207 206 206
query99 1575 1273 1280 1273
Total cold run time: 283004 ms
Total hot run time: 186595 ms
ClickBench: Total hot run time: 31.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b2f87cd3b48345dacc417d984aaa716aff3cf40, data reload: false
query1 0.05 0.04 0.04
query2 0.08 0.04 0.04
query3 0.22 0.06 0.05
query4 1.67 0.07 0.08
query5 0.51 0.50 0.50
query6 1.12 0.73 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.50 0.49
query10 0.54 0.54 0.55
query11 0.16 0.12 0.11
query12 0.15 0.12 0.12
query13 0.61 0.58 0.58
query14 0.76 0.80 0.77
query15 0.83 0.82 0.81
query16 0.38 0.38 0.39
query17 1.07 1.06 1.00
query18 0.21 0.20 0.21
query19 1.86 1.84 1.76
query20 0.01 0.00 0.01
query21 15.43 0.65 0.65
query22 4.45 6.16 2.33
query23 18.27 1.33 1.22
query24 2.06 0.23 0.22
query25 0.15 0.08 0.08
query26 0.26 0.18 0.17
query27 0.08 0.08 0.07
query28 13.32 1.03 1.00
query29 12.60 3.33 3.31
query30 0.24 0.06 0.06
query31 2.87 0.40 0.39
query32 3.25 0.47 0.47
query33 2.97 3.00 3.00
query34 16.87 4.41 4.50
query35 4.47 4.51 4.42
query36 0.66 0.46 0.50
query37 0.19 0.17 0.16
query38 0.17 0.15 0.16
query39 0.05 0.04 0.04
query40 0.16 0.13 0.13
query41 0.09 0.04 0.04
query42 0.06 0.05 0.05
query43 0.04 0.05 0.04
Total cold run time: 109.56 s
Total hot run time: 31.11 s
@superdiaodiao Hello, what 'P0 Regression (Doris Regression) — TeamCity build failed' means? I can not see the details.
@superdiaodiao Hello, what 'P0 Regression (Doris Regression) — TeamCity build failed' means? I can not see the details.
You can click the Details:
then click Login as guest, so that you can see the log.
TPC-H: Total hot run time: 37709 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 598bfa16d37f64101e2b6a098307112f00859234, data reload: false
------ Round 1 ----------------------------------
q1 17626 4419 4290 4290
q2 2021 183 178 178
q3 11927 960 1132 960
q4 10517 675 722 675
q5 7756 2850 2769 2769
q6 225 133 133 133
q7 970 613 602 602
q8 9326 2045 2072 2045
q9 6932 6522 6549 6522
q10 6991 2208 2223 2208
q11 442 238 242 238
q12 388 216 220 216
q13 18741 3057 3027 3027
q14 286 243 239 239
q15 521 478 492 478
q16 507 398 384 384
q17 970 712 622 622
q18 7421 6884 6861 6861
q19 1388 956 960 956
q20 704 346 337 337
q21 4004 3063 2944 2944
q22 1140 1025 1059 1025
Total cold run time: 110803 ms
Total hot run time: 37709 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4337 4288 4381 4288
q2 378 278 280 278
q3 2851 2668 2653 2653
q4 1962 1668 1646 1646
q5 5585 5644 5674 5644
q6 252 139 130 130
q7 2222 1790 1841 1790
q8 3297 3386 3410 3386
q9 8808 8814 8867 8814
q10 3594 3387 3392 3387
q11 583 509 512 509
q12 832 637 657 637
q13 11701 3230 3214 3214
q14 326 294 303 294
q15 530 496 501 496
q16 507 453 432 432
q17 1843 1526 1524 1524
q18 8128 7873 7998 7873
q19 1733 1681 1595 1595
q20 2146 1886 1900 1886
q21 5834 5462 5486 5462
q22 1201 1056 1045 1045
Total cold run time: 68650 ms
Total hot run time: 56983 ms
TPC-DS: Total hot run time: 191420 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 598bfa16d37f64101e2b6a098307112f00859234, data reload: false
query1 1275 876 871 871
query2 6167 1923 1920 1920
query3 10716 4146 4073 4073
query4 59781 23907 23251 23251
query5 5403 511 504 504
query6 404 163 176 163
query7 5755 293 295 293
query8 283 210 206 206
query9 9262 2459 2406 2406
query10 488 284 256 256
query11 17929 15110 15219 15110
query12 160 110 104 104
query13 1571 407 387 387
query14 10531 7134 6957 6957
query15 227 174 170 170
query16 7396 537 496 496
query17 1098 557 553 553
query18 2029 306 287 287
query19 296 150 163 150
query20 123 112 109 109
query21 209 110 101 101
query22 4615 4624 4531 4531
query23 34323 33565 33327 33327
query24 5945 2848 2835 2835
query25 517 375 379 375
query26 679 158 156 156
query27 1785 280 277 277
query28 3742 2036 2037 2036
query29 667 399 400 399
query30 228 145 147 145
query31 953 780 763 763
query32 74 52 60 52
query33 448 281 280 280
query34 845 469 474 469
query35 812 708 718 708
query36 1045 932 944 932
query37 136 83 77 77
query38 3955 3908 3827 3827
query39 1447 1391 1378 1378
query40 190 117 113 113
query41 46 43 43 43
query42 109 97 96 96
query43 508 461 468 461
query44 1074 737 735 735
query45 198 170 168 168
query46 1061 731 739 731
query47 1906 1809 1831 1809
query48 371 298 290 290
query49 779 437 461 437
query50 798 411 418 411
query51 7177 7157 7071 7071
query52 100 89 86 86
query53 251 180 183 180
query54 564 465 457 457
query55 78 78 75 75
query56 282 274 260 260
query57 1184 1101 1107 1101
query58 252 230 258 230
query59 3000 2798 2599 2599
query60 300 289 279 279
query61 149 125 120 120
query62 760 655 659 655
query63 210 188 184 184
query64 3388 1757 1742 1742
query65 3160 3142 3133 3133
query66 670 330 336 330
query67 15440 15158 15256 15158
query68 2981 581 578 578
query69 407 315 292 292
query70 1109 1112 1040 1040
query71 343 271 286 271
query72 2571 2094 2085 2085
query73 700 325 324 324
query74 9130 8945 8809 8809
query75 3355 2693 2710 2693
query76 1343 1011 998 998
query77 526 319 314 314
query78 9587 9444 9031 9031
query79 1029 539 527 527
query80 689 503 493 493
query81 529 231 226 226
query82 286 140 135 135
query83 168 146 147 146
query84 267 79 73 73
query85 663 346 286 286
query86 308 279 299 279
query87 4388 4362 4144 4144
query88 3284 2356 2339 2339
query89 370 282 282 282
query90 1807 190 188 188
query91 122 95 98 95
query92 57 51 52 51
query93 1045 522 526 522
query94 604 297 302 297
query95 347 255 264 255
query96 582 269 280 269
query97 3206 3040 3059 3040
query98 219 211 195 195
query99 1719 1255 1279 1255
Total cold run time: 301065 ms
Total hot run time: 191420 ms
ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 598bfa16d37f64101e2b6a098307112f00859234, data reload: false
query1 0.05 0.04 0.03
query2 0.08 0.03 0.04
query3 0.22 0.05 0.05
query4 1.69 0.07 0.07
query5 0.52 0.50 0.50
query6 1.12 0.75 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.48 0.49
query10 0.55 0.55 0.55
query11 0.15 0.12 0.11
query12 0.14 0.12 0.12
query13 0.61 0.58 0.57
query14 0.76 0.81 0.76
query15 0.87 0.82 0.83
query16 0.39 0.39 0.39
query17 1.03 1.00 1.04
query18 0.21 0.20 0.20
query19 1.94 1.75 1.72
query20 0.01 0.01 0.00
query21 15.42 0.67 0.66
query22 4.40 6.68 2.15
query23 18.28 1.36 1.19
query24 2.15 0.23 0.22
query25 0.15 0.09 0.07
query26 0.26 0.18 0.17
query27 0.07 0.08 0.07
query28 13.23 1.00 1.00
query29 12.60 3.33 3.32
query30 0.25 0.05 0.05
query31 2.88 0.39 0.39
query32 3.28 0.48 0.47
query33 2.94 2.98 3.01
query34 16.83 4.48 4.43
query35 4.50 4.49 4.51
query36 0.66 0.48 0.51
query37 0.20 0.17 0.15
query38 0.15 0.15 0.15
query39 0.05 0.04 0.03
query40 0.16 0.13 0.13
query41 0.10 0.05 0.06
query42 0.06 0.06 0.05
query43 0.05 0.04 0.04
Total cold run time: 109.63 s
Total hot run time: 30.91 s
TPC-H: Total hot run time: 37570 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7061e1cc509b10c141719da35217b1168a191865, data reload: false
------ Round 1 ----------------------------------
q1 17604 4435 4267 4267
q2 2011 180 179 179
q3 11844 964 1161 964
q4 10519 714 756 714
q5 7766 2855 2791 2791
q6 223 137 138 137
q7 984 622 609 609
q8 9306 2053 2058 2053
q9 7131 6547 6534 6534
q10 6995 2163 2190 2163
q11 452 246 246 246
q12 394 221 222 221
q13 18872 3025 3003 3003
q14 273 238 233 233
q15 533 505 494 494
q16 498 388 389 388
q17 980 763 650 650
q18 7379 6886 6766 6766
q19 1393 984 1013 984
q20 673 334 337 334
q21 3891 2845 3065 2845
q22 1108 995 1002 995
Total cold run time: 110829 ms
Total hot run time: 37570 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4332 4338 4285 4285
q2 385 271 274 271
q3 2862 2667 2675 2667
q4 1936 1691 1659 1659
q5 5586 5680 5714 5680
q6 233 132 135 132
q7 2205 1820 1851 1820
q8 3311 3418 3416 3416
q9 8825 8829 8808 8808
q10 3552 3376 3391 3376
q11 597 507 530 507
q12 829 668 700 668
q13 15057 3252 3163 3163
q14 334 288 293 288
q15 553 501 497 497
q16 518 454 460 454
q17 1817 1513 1513 1513
q18 8140 7651 7944 7651
q19 1734 1515 1643 1515
q20 2138 1943 1915 1915
q21 5791 5585 5474 5474
q22 1192 1091 1107 1091
Total cold run time: 71927 ms
Total hot run time: 56850 ms
TPC-DS: Total hot run time: 192399 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7061e1cc509b10c141719da35217b1168a191865, data reload: false
query1 1241 896 869 869
query2 6293 1968 1866 1866
query3 10580 3911 3922 3911
query4 59422 25938 23170 23170
query5 5383 511 485 485
query6 422 166 175 166
query7 5868 307 305 305
query8 284 207 213 207
query9 8725 2464 2445 2445
query10 472 274 257 257
query11 17356 15078 15214 15078
query12 159 104 109 104
query13 1596 385 389 385
query14 11188 7272 6551 6551
query15 247 187 185 185
query16 7587 476 491 476
query17 1154 584 572 572
query18 2018 323 294 294
query19 298 158 149 149
query20 124 115 116 115
query21 209 103 99 99
query22 4477 4419 4349 4349
query23 34263 33786 33412 33412
query24 5850 2901 2832 2832
query25 524 367 384 367
query26 687 153 155 153
query27 1799 281 278 278
query28 3482 2057 2036 2036
query29 688 406 429 406
query30 234 153 159 153
query31 937 728 786 728
query32 81 52 56 52
query33 415 282 284 282
query34 869 482 486 482
query35 838 731 727 727
query36 1059 919 945 919
query37 137 77 77 77
query38 4048 3825 3928 3825
query39 1441 1410 1403 1403
query40 206 115 115 115
query41 46 45 46 45
query42 110 99 98 98
query43 508 471 460 460
query44 1130 744 749 744
query45 195 167 167 167
query46 1096 757 756 756
query47 1877 1813 1777 1777
query48 369 289 297 289
query49 760 435 442 435
query50 810 417 407 407
query51 7245 7102 7067 7067
query52 101 90 95 90
query53 256 181 185 181
query54 565 463 464 463
query55 78 77 77 77
query56 292 265 268 265
query57 1223 1056 1085 1056
query58 226 242 229 229
query59 2989 2893 2969 2893
query60 322 278 307 278
query61 122 122 121 121
query62 753 662 675 662
query63 221 189 188 188
query64 2955 747 736 736
query65 3219 3165 3123 3123
query66 700 342 355 342
query67 15323 15213 15344 15213
query68 3033 580 570 570
query69 419 287 273 273
query70 1200 1132 1147 1132
query71 350 279 282 279
query72 6319 4100 4094 4094
query73 771 342 336 336
query74 9140 8809 8743 8743
query75 3399 2746 2714 2714
query76 1446 1048 1120 1048
query77 571 333 324 324
query78 9777 9129 9541 9129
query79 1586 520 529 520
query80 842 537 527 527
query81 564 223 223 223
query82 286 133 134 133
query83 226 151 155 151
query84 263 74 75 74
query85 852 286 289 286
query86 368 295 289 289
query87 4347 4283 4257 4257
query88 3530 2376 2359 2359
query89 392 284 295 284
query90 2033 193 188 188
query91 123 97 101 97
query92 63 50 53 50
query93 1592 536 536 536
query94 787 299 295 295
query95 354 267 266 266
query96 599 270 269 269
query97 3170 3130 3040 3040
query98 216 203 203 203
query99 1596 1272 1263 1263
Total cold run time: 306366 ms
Total hot run time: 192399 ms
ClickBench: Total hot run time: 30.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7061e1cc509b10c141719da35217b1168a191865, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.04 0.04
query3 0.22 0.04 0.05
query4 1.69 0.08 0.07
query5 0.52 0.51 0.48
query6 1.14 0.74 0.72
query7 0.02 0.02 0.02
query8 0.06 0.05 0.05
query9 0.55 0.49 0.50
query10 0.55 0.55 0.54
query11 0.16 0.11 0.12
query12 0.15 0.12 0.12
query13 0.61 0.59 0.59
query14 0.75 0.81 0.79
query15 0.88 0.81 0.82
query16 0.37 0.40 0.38
query17 1.06 1.01 0.95
query18 0.20 0.20 0.20
query19 1.89 1.73 1.90
query20 0.02 0.01 0.00
query21 15.41 0.65 0.64
query22 4.02 7.75 1.91
query23 18.28 1.38 1.27
query24 2.07 0.23 0.21
query25 0.15 0.08 0.07
query26 0.27 0.18 0.17
query27 0.07 0.08 0.07
query28 13.23 1.02 0.99
query29 12.61 3.32 3.30
query30 0.24 0.05 0.05
query31 2.89 0.40 0.39
query32 3.27 0.47 0.47
query33 2.93 2.95 3.04
query34 17.13 4.42 4.41
query35 4.50 4.45 4.40
query36 0.66 0.49 0.48
query37 0.19 0.16 0.16
query38 0.15 0.15 0.15
query39 0.05 0.04 0.04
query40 0.16 0.12 0.13
query41 0.10 0.05 0.05
query42 0.06 0.06 0.05
query43 0.04 0.05 0.04
Total cold run time: 109.45 s
Total hot run time: 30.56 s
@joker-star-l I noticed that most regression tests are the same type: func(col, 10). Can we add more case types in current version? Like func(col1, col2) and func(col, another number: 100)?
@joker-star-l I noticed that most regression tests are the same type: func(col, 10). Can we add more case types in current version? Like func(col1, col2) and func(col, another number: 100)?
OK.
TPC-H: Total hot run time: 37928 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 70c272849a9f52516ac6cb7552d31217c75cbf3e, data reload: false
------ Round 1 ----------------------------------
q1 17603 4411 4302 4302
q2 2020 184 177 177
q3 11748 941 1135 941
q4 10514 751 718 718
q5 7772 2839 2793 2793
q6 230 138 134 134
q7 947 615 597 597
q8 9357 2041 2043 2041
q9 7015 6503 6503 6503
q10 7001 2245 2208 2208
q11 485 248 237 237
q12 400 217 225 217
q13 18111 3040 3026 3026
q14 282 240 240 240
q15 528 490 493 490
q16 602 516 524 516
q17 970 613 788 613
q18 7235 6909 6889 6889
q19 1402 1038 1107 1038
q20 700 333 344 333
q21 4033 2911 2904 2904
q22 1105 1045 1011 1011
Total cold run time: 110060 ms
Total hot run time: 37928 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4365 4274 4258 4258
q2 390 275 281 275
q3 2875 2701 2637 2637
q4 1923 1680 1657 1657
q5 5562 5684 5614 5614
q6 222 136 139 136
q7 2253 1828 1856 1828
q8 3309 3411 3412 3411
q9 8824 8738 8753 8738
q10 3567 3323 3389 3323
q11 609 519 541 519
q12 829 624 664 624
q13 14548 3187 3211 3187
q14 340 308 305 305
q15 548 499 477 477
q16 641 567 570 567
q17 1835 1549 1528 1528
q18 8049 7911 7880 7880
q19 1727 1612 1597 1597
q20 2148 1917 1912 1912
q21 5759 5575 5471 5471
q22 1143 1046 1081 1046
Total cold run time: 71466 ms
Total hot run time: 56990 ms
TPC-DS: Total hot run time: 192529 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 70c272849a9f52516ac6cb7552d31217c75cbf3e, data reload: false
query1 1257 906 867 867
query2 6309 2022 1883 1883
query3 10615 3956 3905 3905
query4 59598 27037 23226 23226
query5 5335 510 475 475
query6 398 174 158 158
query7 5785 309 311 309
query8 293 215 215 215
query9 9101 2497 2495 2495
query10 478 275 277 275
query11 18326 15197 15688 15197
query12 168 110 100 100
query13 1582 393 377 377
query14 10749 6958 7411 6958
query15 232 181 178 178
query16 7560 517 511 511
query17 1152 597 570 570
query18 2033 301 296 296
query19 280 146 142 142
query20 127 108 106 106
query21 203 112 102 102
query22 4531 4443 4483 4443
query23 34793 33478 33497 33478
query24 5915 2867 2842 2842
query25 513 382 387 382
query26 671 158 157 157
query27 1757 282 279 279
query28 3823 2120 2101 2101
query29 641 406 414 406
query30 245 152 157 152
query31 954 771 741 741
query32 81 54 53 53
query33 445 280 279 279
query34 877 484 489 484
query35 852 729 797 729
query36 1086 941 937 937
query37 144 103 99 99
query38 3986 3844 3897 3844
query39 1457 1383 1380 1380
query40 205 122 117 117
query41 47 48 47 47
query42 113 100 100 100
query43 526 486 489 486
query44 1096 749 750 749
query45 200 171 166 166
query46 1088 781 756 756
query47 1893 1808 1796 1796
query48 385 307 311 307
query49 774 452 439 439
query50 831 415 418 415
query51 7140 7050 7033 7033
query52 97 87 95 87
query53 249 179 181 179
query54 555 454 454 454
query55 79 76 78 76
query56 272 247 242 242
query57 1172 1055 1099 1055
query58 229 221 226 221
query59 3037 2898 2690 2690
query60 288 264 276 264
query61 101 97 126 97
query62 760 665 645 645
query63 212 184 184 184
query64 2676 657 658 657
query65 3267 3153 3129 3129
query66 652 332 324 324
query67 15425 15497 15130 15130
query68 3020 574 582 574
query69 404 275 278 275
query70 1168 1008 1168 1008
query71 358 293 279 279
query72 5155 3963 4284 3963
query73 742 335 333 333
query74 9064 8817 8818 8817
query75 3399 2713 2681 2681
query76 1370 1026 1010 1010
query77 535 322 319 319
query78 9653 9056 9037 9037
query79 1056 540 534 534
query80 695 516 499 499
query81 525 233 235 233
query82 250 141 143 141
query83 173 148 148 148
query84 262 76 79 76
query85 678 279 316 279
query86 321 302 297 297
query87 4302 4342 4168 4168
query88 3225 2355 2350 2350
query89 378 297 288 288
query90 1946 201 196 196
query91 214 102 101 101
query92 62 53 51 51
query93 1052 545 551 545
query94 690 302 307 302
query95 326 272 266 266
query96 591 275 271 271
query97 3176 3058 3113 3058
query98 213 199 209 199
query99 1559 1255 1260 1255
Total cold run time: 304283 ms
Total hot run time: 192529 ms
ClickBench: Total hot run time: 31.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 70c272849a9f52516ac6cb7552d31217c75cbf3e, data reload: false
query1 0.05 0.04 0.04
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.65 0.09 0.11
query5 0.50 0.49 0.48
query6 1.12 0.73 0.73
query7 0.02 0.02 0.01
query8 0.04 0.04 0.04
query9 0.56 0.49 0.49
query10 0.55 0.54 0.55
query11 0.16 0.11 0.11
query12 0.15 0.12 0.12
query13 0.62 0.59 0.59
query14 2.04 2.04 2.05
query15 0.85 0.81 0.82
query16 0.38 0.38 0.36
query17 0.96 1.01 1.02
query18 0.20 0.20 0.21
query19 1.87 1.79 1.76
query20 0.01 0.00 0.01
query21 15.39 0.67 0.65
query22 4.70 7.31 1.51
query23 18.31 1.42 1.25
query24 2.10 0.22 0.23
query25 0.16 0.09 0.08
query26 0.26 0.18 0.18
query27 0.08 0.08 0.08
query28 13.24 1.01 0.99
query29 12.62 3.47 3.40
query30 0.24 0.05 0.06
query31 2.89 0.41 0.40
query32 3.23 0.47 0.48
query33 3.00 2.97 3.03
query34 16.98 4.36 4.44
query35 4.42 4.43 4.47
query36 0.65 0.49 0.46
query37 0.19 0.15 0.16
query38 0.16 0.15 0.15
query39 0.05 0.04 0.03
query40 0.16 0.13 0.13
query41 0.10 0.05 0.05
query42 0.06 0.05 0.06
query43 0.05 0.05 0.04
Total cold run time: 111.08 s
Total hot run time: 31.62 s