doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[branch-2.0](function) fix some date functions (#26475)(#35080)(#35104)
fix overflow of date_add function
Proposed changes
Issue Number: close #xxx
pick https://github.com/apache/doris/pull/26475, https://github.com/apache/doris/pull/35080 and https://github.com/apache/doris/pull/35104
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TPC-H: Total hot run time: 50120 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d53c38cac99008b96f6cad2179f82dad81f74eea, data reload: false
------ Round 1 ----------------------------------
q1 17972 4411 4350 4350
q2 2025 155 147 147
q3 10411 1921 1965 1921
q4 10271 1321 1351 1321
q5 8713 3934 3941 3934
q6 239 126 129 126
q7 2100 1592 1614 1592
q8 9319 2749 2721 2721
q9 11083 10417 10428 10417
q10 8632 3506 3530 3506
q11 418 235 253 235
q12 471 302 305 302
q13 18381 3972 4055 3972
q14 355 336 319 319
q15 501 456 460 456
q16 694 564 571 564
q17 1134 1007 979 979
q18 7358 6885 6859 6859
q19 1723 1616 1566 1566
q20 523 308 309 308
q21 4460 4155 4157 4155
q22 474 389 370 370
Total cold run time: 117257 ms
Total hot run time: 50120 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4343 4319 4280 4280
q2 318 219 223 219
q3 4206 4125 4115 4115
q4 2768 2762 2745 2745
q5 7216 7147 7109 7109
q6 235 122 118 118
q7 3174 2871 2800 2800
q8 4343 4435 4538 4435
q9 17507 16959 17064 16959
q10 4226 4282 4261 4261
q11 736 673 675 673
q12 1028 857 856 856
q13 7087 3736 3743 3736
q14 456 429 430 429
q15 497 453 451 451
q16 730 673 685 673
q17 3905 3835 3826 3826
q18 8759 8748 8642 8642
q19 1724 1707 1654 1654
q20 2343 2144 2094 2094
q21 8471 8498 8356 8356
q22 1015 978 978 978
Total cold run time: 85087 ms
Total hot run time: 79409 ms
TeamCity be ut coverage result: Function Coverage: 37.79% (8076/21369) Line Coverage: 29.46% (65967/223934) Region Coverage: 28.92% (33953/117408) Branch Coverage: 24.77% (17424/70334) Coverage Report: http://coverage.selectdb-in.cc/coverage/d53c38cac99008b96f6cad2179f82dad81f74eea_d53c38cac99008b96f6cad2179f82dad81f74eea/report/index.html
TPC-DS: Total hot run time: 203281 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d53c38cac99008b96f6cad2179f82dad81f74eea, data reload: false
query1 945 389 378 378
query2 6553 2818 2534 2534
query3 6915 214 212 212
query4 21190 17906 17895 17895
query5 19736 6544 6439 6439
query6 283 218 230 218
query7 4151 299 307 299
query8 287 231 244 231
query9 3180 2715 2632 2632
query10 419 305 309 305
query11 11351 10774 10773 10773
query12 129 81 75 75
query13 5584 694 679 679
query14 17716 13342 13312 13312
query15 367 218 226 218
query16 6468 276 262 262
query17 1706 1450 867 867
query18 2336 412 400 400
query19 210 145 155 145
query20 70 78 84 78
query21 188 95 94 94
query22 5315 5088 5036 5036
query23 32761 31878 31817 31817
query24 6878 6483 6570 6483
query25 524 421 411 411
query26 531 164 164 164
query27 1881 293 293 293
query28 6193 2359 2305 2305
query29 2894 2915 2744 2744
query30 237 161 164 161
query31 916 730 697 697
query32 72 62 59 59
query33 408 265 253 253
query34 844 479 487 479
query35 1147 925 876 876
query36 1239 1248 1134 1134
query37 88 60 58 58
query38 3089 2944 2996 2944
query39 1361 1328 1339 1328
query40 200 93 90 90
query41 38 40 37 37
query42 87 85 84 84
query43 703 721 720 720
query44 1123 713 716 713
query45 236 234 226 226
query46 1245 971 982 971
query47 1792 1702 1619 1619
query48 1015 710 695 695
query49 616 355 378 355
query50 865 619 632 619
query51 4771 4643 4711 4643
query52 85 77 80 77
query53 451 324 324 324
query54 2674 2464 2467 2464
query55 89 84 80 80
query56 237 220 221 220
query57 1260 1162 1143 1143
query58 213 190 202 190
query59 4210 4054 3722 3722
query60 217 193 197 193
query61 86 85 86 85
query62 862 497 444 444
query63 490 343 340 340
query64 2417 1507 1486 1486
query65 3621 3841 3526 3526
query66 760 373 370 370
query67 16138 15541 15506 15506
query68 8712 649 667 649
query69 587 343 352 343
query70 1721 1251 1361 1251
query71 414 292 310 292
query72 6445 3570 3510 3510
query73 739 327 318 318
query74 6381 5857 5881 5857
query75 5290 3714 3764 3714
query76 5336 1139 1162 1139
query77 898 250 260 250
query78 12671 11977 12138 11977
query79 7243 634 673 634
query80 939 390 395 390
query81 493 231 235 231
query82 1540 102 97 97
query83 167 137 134 134
query84 259 68 70 68
query85 860 305 301 301
query86 317 288 307 288
query87 3244 2982 2980 2980
query88 4803 2371 2382 2371
query89 380 299 296 296
query90 1879 200 214 200
query91 172 132 134 132
query92 60 51 50 50
query93 4972 567 581 567
query94 639 203 208 203
query95 1113 1052 1052 1052
query96 643 335 339 335
query97 6507 6399 6423 6399
query98 199 185 178 178
query99 2953 845 1040 845
Total cold run time: 311239 ms
Total hot run time: 203281 ms
ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d53c38cac99008b96f6cad2179f82dad81f74eea, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.02 0.02
query3 0.26 0.05 0.04
query4 1.79 0.07 0.06
query5 0.55 0.53 0.52
query6 1.31 0.60 0.62
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.53 0.46 0.48
query10 0.53 0.54 0.53
query11 0.12 0.08 0.08
query12 0.13 0.09 0.09
query13 0.61 0.62 0.61
query14 0.79 0.79 0.78
query15 0.77 0.74 0.75
query16 0.37 0.38 0.37
query17 1.01 1.02 1.02
query18 0.23 0.25 0.26
query19 1.93 1.83 1.84
query20 0.02 0.01 0.01
query21 15.63 0.53 0.54
query22 2.32 2.22 1.63
query23 17.37 1.07 1.11
query24 5.07 1.04 1.12
query25 0.37 0.11 0.05
query26 0.55 0.16 0.15
query27 0.04 0.05 0.04
query28 7.88 0.81 0.73
query29 12.72 2.36 2.29
query30 0.60 0.55 0.55
query31 2.81 0.38 0.37
query32 3.38 0.50 0.51
query33 3.05 3.08 3.07
query34 15.24 4.80 4.79
query35 4.86 4.87 4.87
query36 1.05 1.01 1.01
query37 0.06 0.05 0.04
query38 0.04 0.02 0.02
query39 0.02 0.01 0.02
query40 0.16 0.14 0.14
query41 0.06 0.01 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 104.42 s
Total hot run time: 31 s
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Load test result on commit d53c38cac99008b96f6cad2179f82dad81f74eea with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.3 seconds inserted 10000000 Rows, about 469K ops/s
TPC-H: Total hot run time: 49762 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a, data reload: false
------ Round 1 ----------------------------------
q1 17745 4430 4348 4348
q2 2058 156 144 144
q3 10528 1855 1925 1855
q4 10334 1293 1321 1293
q5 8552 3914 3929 3914
q6 226 128 122 122
q7 2003 1612 1635 1612
q8 9518 2722 2695 2695
q9 14251 10351 10303 10303
q10 8645 3546 3492 3492
q11 416 229 227 227
q12 463 303 298 298
q13 18331 3942 3995 3942
q14 367 330 333 330
q15 502 459 473 459
q16 672 573 569 569
q17 1127 937 934 934
q18 7262 6949 6996 6949
q19 1720 1537 1527 1527
q20 543 310 296 296
q21 4444 4116 4054 4054
q22 499 399 400 399
Total cold run time: 120206 ms
Total hot run time: 49762 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4300 4288 4269 4269
q2 313 221 218 218
q3 4157 4119 4124 4119
q4 2737 2760 2740 2740
q5 7243 7084 7099 7084
q6 239 120 122 120
q7 3219 2824 2786 2786
q8 4352 4458 4447 4447
q9 17345 17090 16994 16994
q10 4189 4276 4236 4236
q11 760 714 675 675
q12 1016 858 836 836
q13 7323 3718 3753 3718
q14 451 425 427 425
q15 505 457 458 457
q16 731 699 676 676
q17 3757 3806 3787 3787
q18 8697 8800 8808 8800
q19 1722 1696 1639 1639
q20 2365 2130 2081 2081
q21 8544 8390 8424 8390
q22 1019 933 915 915
Total cold run time: 84984 ms
Total hot run time: 79412 ms
TeamCity be ut coverage result: Function Coverage: 37.79% (8076/21369) Line Coverage: 29.45% (65962/223955) Region Coverage: 28.91% (33951/117417) Branch Coverage: 24.77% (17422/70336) Coverage Report: http://coverage.selectdb-in.cc/coverage/a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a_a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a/report/index.html
TPC-DS: Total hot run time: 202941 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a, data reload: false
query1 923 382 382 382
query2 6550 2695 2456 2456
query3 6923 207 211 207
query4 19963 17963 17876 17876
query5 19733 6496 6522 6496
query6 283 218 228 218
query7 4155 293 307 293
query8 269 269 245 245
query9 3099 2685 2607 2607
query10 433 290 293 290
query11 11291 10551 10663 10551
query12 115 80 71 71
query13 5576 688 679 679
query14 18101 13721 13314 13314
query15 361 237 226 226
query16 6440 272 253 253
query17 1726 1464 853 853
query18 2327 405 404 404
query19 214 146 148 146
query20 73 79 78 78
query21 194 94 99 94
query22 5275 4952 5044 4952
query23 32593 31904 31702 31702
query24 6874 6447 6459 6447
query25 534 418 410 410
query26 529 163 160 160
query27 1896 300 291 291
query28 6119 2343 2315 2315
query29 2813 2869 2693 2693
query30 233 161 165 161
query31 898 724 725 724
query32 70 63 58 58
query33 397 253 236 236
query34 843 474 486 474
query35 1142 945 935 935
query36 1188 1490 1335 1335
query37 88 58 63 58
query38 3074 2916 2943 2916
query39 1379 1326 1335 1326
query40 201 90 96 90
query41 39 40 35 35
query42 78 85 88 85
query43 695 682 695 682
query44 1115 709 725 709
query45 243 231 226 226
query46 1214 950 975 950
query47 1801 1749 1755 1749
query48 1009 735 718 718
query49 628 382 383 382
query50 864 590 575 575
query51 4770 4646 4698 4646
query52 98 87 86 86
query53 437 314 315 314
query54 2672 2445 2460 2445
query55 87 86 85 85
query56 234 224 201 201
query57 1179 1090 1088 1088
query58 210 192 191 191
query59 4084 4021 3931 3931
query60 206 183 214 183
query61 89 85 86 85
query62 888 455 526 455
query63 489 334 338 334
query64 2331 1496 1351 1351
query65 3610 3583 3524 3524
query66 798 371 363 363
query67 16323 15560 15524 15524
query68 8501 642 660 642
query69 556 363 351 351
query70 1545 1335 1424 1335
query71 404 307 321 307
query72 6427 3508 3414 3414
query73 730 327 327 327
query74 6226 5911 5775 5775
query75 5262 3698 3701 3698
query76 5140 1171 1206 1171
query77 872 257 258 257
query78 12778 12259 11921 11921
query79 13582 634 637 634
query80 897 392 383 383
query81 487 229 232 229
query82 1228 95 96 95
query83 170 132 133 132
query84 263 68 71 68
query85 807 305 297 297
query86 346 324 299 299
query87 3214 3013 2980 2980
query88 6384 2388 2365 2365
query89 452 284 284 284
query90 2550 211 221 211
query91 174 133 137 133
query92 62 53 53 53
query93 6676 584 590 584
query94 1271 203 206 203
query95 1113 1063 1069 1063
query96 643 332 336 332
query97 6576 6410 6507 6410
query98 190 175 173 173
query99 2953 876 971 876
Total cold run time: 319565 ms
Total hot run time: 202941 ms
ClickBench: Total hot run time: 30.55 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a, data reload: false
query1 0.02 0.02 0.01
query2 0.08 0.02 0.02
query3 0.25 0.04 0.04
query4 1.78 0.06 0.07
query5 0.54 0.53 0.52
query6 1.23 0.65 0.66
query7 0.01 0.01 0.01
query8 0.04 0.02 0.03
query9 0.53 0.47 0.48
query10 0.54 0.52 0.52
query11 0.12 0.08 0.08
query12 0.12 0.08 0.08
query13 0.61 0.60 0.60
query14 0.80 0.77 0.80
query15 0.79 0.76 0.77
query16 0.37 0.36 0.38
query17 0.98 1.02 1.02
query18 0.22 0.26 0.25
query19 1.85 1.84 1.82
query20 0.02 0.01 0.01
query21 15.48 0.54 0.54
query22 2.10 2.30 1.58
query23 17.25 1.09 0.88
query24 6.16 0.86 1.35
query25 0.38 0.07 0.09
query26 0.74 0.16 0.15
query27 0.03 0.04 0.04
query28 6.77 0.71 0.69
query29 12.58 2.33 2.32
query30 0.61 0.54 0.53
query31 2.80 0.40 0.37
query32 3.35 0.49 0.50
query33 3.07 3.07 3.05
query34 15.27 4.83 4.82
query35 4.87 4.84 4.87
query36 1.07 1.00 1.03
query37 0.06 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.01 0.01
query40 0.16 0.14 0.14
query41 0.07 0.01 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.01
Total cold run time: 103.81 s
Total hot run time: 30.55 s
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Load test result on commit a7308f56c0bf277bd13c53f94c8a00d0ca5eab1a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s