doris
doris copied to clipboard
[fix](Export) fix some issues of Export
Proposed changes
Issue Number: close #xxx
-
Forbid rollback to the old optimizer in
Export
task. Since originStmt is empty, reverting to the old optimizer when the new optimizer is enabled is meaningless. -
Display
parallelism
in 'show export'. -
Create an initial Map for resultAttachedInfo to avoid NullPointerException.
-
Remove the originStmt added in this PR #31020, because the
Export
statement is underlying the Outfile statement, it cannot be treated asOriginStmt
.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
run buildall
TPC-H: Total hot run time: 41319 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 69d60829beabbd91463557728787d336af5c3e86, data reload: false
------ Round 1 ----------------------------------
q1 17593 4362 4307 4307
q2 2018 188 189 188
q3 10450 1308 1239 1239
q4 10190 831 796 796
q5 7493 2734 2748 2734
q6 246 134 134 134
q7 1038 631 598 598
q8 9235 2162 2110 2110
q9 9523 6748 6722 6722
q10 9301 3929 3929 3929
q11 445 252 257 252
q12 455 224 246 224
q13 18041 3159 3212 3159
q14 251 220 215 215
q15 519 464 475 464
q16 513 422 396 396
q17 1011 678 658 658
q18 8278 7724 7676 7676
q19 3280 1585 1555 1555
q20 634 351 334 334
q21 5183 4473 3355 3355
q22 352 274 288 274
Total cold run time: 116049 ms
Total hot run time: 41319 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4537 4407 4370 4370
q2 369 281 267 267
q3 3168 2942 2884 2884
q4 2034 1595 1635 1595
q5 5377 5520 5457 5457
q6 215 124 124 124
q7 2359 1951 1966 1951
q8 3292 3444 3410 3410
q9 8639 8783 8813 8783
q10 4096 3734 3818 3734
q11 593 495 490 490
q12 807 631 624 624
q13 15986 3135 3128 3128
q14 290 270 278 270
q15 529 481 494 481
q16 471 429 456 429
q17 1791 1518 1517 1517
q18 7662 7508 7319 7319
q19 1639 1475 1524 1475
q20 1945 1784 1764 1764
q21 8468 4933 4909 4909
q22 575 496 508 496
Total cold run time: 74842 ms
Total hot run time: 55477 ms
TPC-DS: Total hot run time: 186701 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 69d60829beabbd91463557728787d336af5c3e86, data reload: false
query1 920 374 347 347
query2 6442 2310 2339 2310
query3 6642 213 216 213
query4 22967 21122 21206 21122
query5 4096 408 413 408
query6 271 186 184 184
query7 4575 285 297 285
query8 245 188 184 184
query9 8507 2432 2424 2424
query10 439 247 260 247
query11 14571 14184 14052 14052
query12 140 92 88 88
query13 1643 371 375 371
query14 10831 7936 7479 7479
query15 255 178 166 166
query16 8063 269 270 269
query17 1712 578 578 578
query18 2095 281 278 278
query19 289 162 153 153
query20 100 92 88 88
query21 201 128 124 124
query22 5068 4939 4863 4863
query23 33899 33196 33178 33178
query24 6524 2942 2824 2824
query25 585 385 380 380
query26 699 151 149 149
query27 1937 318 336 318
query28 4152 2072 2079 2072
query29 850 620 621 620
query30 250 162 153 153
query31 965 745 755 745
query32 91 55 58 55
query33 478 264 255 255
query34 872 476 478 476
query35 759 687 659 659
query36 1034 944 900 900
query37 109 68 67 67
query38 3144 2971 3006 2971
query39 1593 1524 1562 1524
query40 201 126 126 126
query41 45 41 42 41
query42 107 99 97 97
query43 582 548 539 539
query44 1055 734 739 734
query45 268 263 235 235
query46 1055 736 734 734
query47 1955 1875 1875 1875
query48 386 309 306 306
query49 930 388 395 388
query50 780 378 388 378
query51 6790 6784 6668 6668
query52 103 92 91 91
query53 358 295 287 287
query54 264 237 226 226
query55 79 71 72 71
query56 239 224 223 223
query57 1170 1157 1116 1116
query58 212 200 196 196
query59 3381 3238 3098 3098
query60 252 235 238 235
query61 107 90 88 88
query62 582 444 457 444
query63 302 278 279 278
query64 8280 7216 7201 7201
query65 3101 3043 3032 3032
query66 807 344 351 344
query67 15868 15474 15022 15022
query68 9558 558 556 556
query69 608 322 328 322
query70 1377 1081 1110 1081
query71 461 268 285 268
query72 8300 2553 2370 2370
query73 1604 331 337 331
query74 6489 6130 6031 6031
query75 4557 2650 2602 2602
query76 5382 906 1060 906
query77 704 266 265 265
query78 11180 10199 10084 10084
query79 12645 522 523 522
query80 2427 432 423 423
query81 503 241 218 218
query82 666 96 93 93
query83 211 161 162 161
query84 269 86 86 86
query85 1183 274 257 257
query86 398 296 279 279
query87 3289 3065 3099 3065
query88 5206 2429 2414 2414
query89 530 382 372 372
query90 2382 187 182 182
query91 129 94 97 94
query92 57 46 47 46
query93 7347 517 513 513
query94 1496 189 176 176
query95 1095 1088 1088 1088
query96 599 266 268 266
query97 3113 2917 2918 2917
query98 229 224 217 217
query99 1242 886 844 844
Total cold run time: 302508 ms
Total hot run time: 186701 ms
run buildall
run buildall
run buildall
TPC-H: Total hot run time: 40939 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a4e317f09ec573819911effb700829ec3c20db41, data reload: false
------ Round 1 ----------------------------------
q1 17618 4312 4725 4312
q2 2021 187 191 187
q3 10984 1226 1216 1216
q4 10769 833 905 833
q5 7487 2723 2666 2666
q6 223 133 136 133
q7 1014 615 589 589
q8 9576 2120 2103 2103
q9 9207 6651 6595 6595
q10 9022 3697 3688 3688
q11 484 239 231 231
q12 399 224 217 217
q13 18553 2934 2945 2934
q14 255 228 222 222
q15 526 485 471 471
q16 514 377 381 377
q17 958 710 747 710
q18 8031 7460 7520 7460
q19 5217 1491 1467 1467
q20 687 310 301 301
q21 5122 3945 3968 3945
q22 360 282 311 282
Total cold run time: 119027 ms
Total hot run time: 40939 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4329 4266 4218 4218
q2 357 263 263 263
q3 2979 2757 2694 2694
q4 1871 1623 1618 1618
q5 5289 5266 5268 5266
q6 207 124 125 124
q7 2263 1902 1872 1872
q8 3193 3325 3331 3325
q9 8300 8303 8322 8303
q10 3893 3682 3645 3645
q11 573 490 502 490
q12 747 586 592 586
q13 16524 2920 2946 2920
q14 299 279 247 247
q15 514 472 453 453
q16 472 418 417 417
q17 1772 1534 1470 1470
q18 7578 7513 7438 7438
q19 1646 1553 1511 1511
q20 1935 1732 1775 1732
q21 5147 4980 4888 4888
q22 575 479 489 479
Total cold run time: 70463 ms
Total hot run time: 53959 ms
TPC-DS: Total hot run time: 187156 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a4e317f09ec573819911effb700829ec3c20db41, data reload: false
query1 914 358 348 348
query2 6446 2384 2230 2230
query3 6659 206 214 206
query4 23158 21223 21457 21223
query5 4163 421 436 421
query6 264 172 176 172
query7 4583 300 286 286
query8 244 187 185 185
query9 8590 2413 2413 2413
query10 442 253 277 253
query11 14712 14258 14071 14071
query12 147 93 88 88
query13 1638 379 381 379
query14 10132 8439 8528 8439
query15 268 166 168 166
query16 8153 258 265 258
query17 1874 558 544 544
query18 2176 295 267 267
query19 223 149 150 149
query20 92 87 83 83
query21 200 129 128 128
query22 5020 4896 4770 4770
query23 34165 33503 33444 33444
query24 11925 2940 2885 2885
query25 643 367 362 362
query26 1746 158 155 155
query27 2946 319 322 319
query28 7572 2053 2039 2039
query29 1002 619 603 603
query30 299 146 152 146
query31 958 742 744 742
query32 93 59 55 55
query33 746 265 258 258
query34 997 469 476 469
query35 790 694 666 666
query36 1061 884 927 884
query37 184 68 68 68
query38 2883 2752 2734 2734
query39 1646 1614 1557 1557
query40 275 123 129 123
query41 40 39 38 38
query42 108 95 97 95
query43 581 535 529 529
query44 1215 724 739 724
query45 269 225 254 225
query46 1079 703 747 703
query47 1943 1845 1886 1845
query48 359 299 300 299
query49 1194 388 404 388
query50 774 385 389 385
query51 6815 6653 6741 6653
query52 106 91 92 91
query53 346 277 288 277
query54 1018 422 434 422
query55 79 73 73 73
query56 233 223 226 223
query57 1230 1145 1126 1126
query58 219 207 206 206
query59 3413 3145 3308 3145
query60 251 240 239 239
query61 96 89 89 89
query62 670 462 471 462
query63 319 284 285 284
query64 9726 7389 7356 7356
query65 3162 3110 3099 3099
query66 1393 344 344 344
query67 15183 14778 14820 14778
query68 5492 549 537 537
query69 541 290 294 290
query70 1120 1159 1148 1148
query71 448 266 273 266
query72 7748 2550 2361 2361
query73 711 321 323 321
query74 6560 6113 6059 6059
query75 3722 2578 2614 2578
query76 3257 931 936 931
query77 603 267 258 258
query78 10624 10206 10042 10042
query79 2217 512 516 512
query80 1198 448 443 443
query81 511 217 218 217
query82 698 93 93 93
query83 195 164 165 164
query84 265 89 88 88
query85 1439 276 324 276
query86 454 327 313 313
query87 3300 3114 3114 3114
query88 3929 2327 2335 2327
query89 460 381 382 381
query90 1934 192 186 186
query91 127 99 95 95
query92 57 52 52 52
query93 2032 520 513 513
query94 1148 186 184 184
query95 401 304 300 300
query96 586 270 264 264
query97 3198 3018 3012 3012
query98 252 219 223 219
query99 1320 926 939 926
Total cold run time: 288397 ms
Total hot run time: 187156 ms
PR approved by at least one committer and no changes requested.
PR approved by anyone and no changes requested.