doris icon indicating copy to clipboard operation
doris copied to clipboard

[Enhancement] (doris-future) Support auto partition name function

Open wyxxxcat opened this issue 9 months ago • 3 comments

Proposed changes

Issue Number: #34130

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

wyxxxcat avatar Apr 28 '24 12:04 wyxxxcat

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar Apr 28 '24 12:04 doris-robot

run buildall

wyxxxcat avatar May 01 '24 15:05 wyxxxcat

TeamCity be ut coverage result: Function Coverage: 35.68% (8959/25108) Line Coverage: 27.27% (73887/270966) Region Coverage: 26.45% (38173/144295) Branch Coverage: 23.21% (19445/83782) Coverage Report: http://coverage.selectdb-in.cc/coverage/0a1e4b52d276a73d138ddd0592f4e4cee812de5f_0a1e4b52d276a73d138ddd0592f4e4cee812de5f/report/index.html

doris-robot avatar May 01 '24 16:05 doris-robot

run buildall

wyxxxcat avatar May 10 '24 11:05 wyxxxcat

TeamCity be ut coverage result: Function Coverage: 35.67% (8982/25182) Line Coverage: 27.33% (74255/271702) Region Coverage: 26.56% (38387/144508) Branch Coverage: 23.39% (19579/83718) Coverage Report: http://coverage.selectdb-in.cc/coverage/3fb2f9867b64c0a5ee2c8fa8a63b4c60c9eff494_3fb2f9867b64c0a5ee2c8fa8a63b4c60c9eff494/report/index.html

doris-robot avatar May 10 '24 12:05 doris-robot

run buildall

wyxxxcat avatar May 13 '24 02:05 wyxxxcat

TeamCity be ut coverage result: Function Coverage: 35.65% (8987/25210) Line Coverage: 27.31% (74279/272017) Region Coverage: 26.54% (38394/144674) Branch Coverage: 23.34% (19568/83834) Coverage Report: http://coverage.selectdb-in.cc/coverage/3fb2f9867b64c0a5ee2c8fa8a63b4c60c9eff494_3fb2f9867b64c0a5ee2c8fa8a63b4c60c9eff494/report/index.html

doris-robot avatar May 13 '24 03:05 doris-robot

run buildall

wyxxxcat avatar May 14 '24 08:05 wyxxxcat

TeamCity be ut coverage result: Function Coverage: 35.66% (8983/25189) Line Coverage: 27.32% (74266/271887) Region Coverage: 26.55% (38374/144545) Branch Coverage: 23.37% (19575/83762) Coverage Report: http://coverage.selectdb-in.cc/coverage/7a45f9511a3258441d34d724ae932e744f6a4b6b_7a45f9511a3258441d34d724ae932e744f6a4b6b/report/index.html

doris-robot avatar May 14 '24 08:05 doris-robot

run buildall

wyxxxcat avatar May 14 '24 13:05 wyxxxcat

TeamCity be ut coverage result: Function Coverage: 35.65% (8984/25199) Line Coverage: 27.31% (74270/271932) Region Coverage: 26.54% (38378/144587) Branch Coverage: 23.36% (19576/83784) Coverage Report: http://coverage.selectdb-in.cc/coverage/61e62d636379dcf9007e140dc0cadbecd99a02f0_61e62d636379dcf9007e140dc0cadbecd99a02f0/report/index.html

doris-robot avatar May 14 '24 13:05 doris-robot

run buildall

wyxxxcat avatar Jul 17 '24 03:07 wyxxxcat

TPC-H: Total hot run time: 41368 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b156df0619f6ea95364534f7e25482e0262409e7, data reload: false

------ Round 1 ----------------------------------
q1	17613	4499	4465	4465
q2	2004	206	188	188
q3	10428	1284	1142	1142
q4	10188	862	816	816
q5	7612	2908	2837	2837
q6	245	138	145	138
q7	1043	631	622	622
q8	9229	2280	2245	2245
q9	8912	6853	6826	6826
q10	8843	3872	3892	3872
q11	472	242	252	242
q12	425	224	233	224
q13	18677	2996	2994	2994
q14	291	244	246	244
q15	548	501	491	491
q16	517	402	382	382
q17	1018	699	655	655
q18	8150	7533	7541	7533
q19	5598	1557	1468	1468
q20	673	343	319	319
q21	5148	3373	4066	3373
q22	361	299	292	292
Total cold run time: 117995 ms
Total hot run time: 41368 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4588	4437	4420	4420
q2	394	271	289	271
q3	3276	3021	2953	2953
q4	2101	1757	1727	1727
q5	5639	5627	5670	5627
q6	244	141	133	133
q7	2338	1906	1845	1845
q8	3545	3723	3678	3678
q9	8891	9000	8875	8875
q10	4133	3950	3999	3950
q11	629	500	492	492
q12	808	635	643	635
q13	17050	3232	3254	3232
q14	333	300	287	287
q15	574	482	525	482
q16	500	435	455	435
q17	1941	1606	1586	1586
q18	8103	8058	7835	7835
q19	1864	1826	1724	1724
q20	2207	1901	1891	1891
q21	5291	5092	5039	5039
q22	611	523	536	523
Total cold run time: 75060 ms
Total hot run time: 57640 ms

doris-robot avatar Jul 17 '24 04:07 doris-robot

TPC-DS: Total hot run time: 173762 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b156df0619f6ea95364534f7e25482e0262409e7, data reload: false

query1	906	376	366	366
query2	6362	1860	1840	1840
query3	6665	207	220	207
query4	25727	17769	17300	17300
query5	3823	475	470	470
query6	272	168	164	164
query7	4584	298	290	290
query8	226	188	184	184
query9	8533	2392	2381	2381
query10	443	305	271	271
query11	12185	10041	10101	10041
query12	118	87	82	82
query13	1642	378	386	378
query14	10049	7962	7735	7735
query15	221	173	173	173
query16	7486	327	324	324
query17	1779	563	538	538
query18	1599	290	285	285
query19	206	158	155	155
query20	93	88	85	85
query21	211	129	125	125
query22	4372	3989	3969	3969
query23	34007	34101	33744	33744
query24	10689	3026	2952	2952
query25	587	376	383	376
query26	710	149	149	149
query27	2213	282	289	282
query28	6073	2092	2046	2046
query29	855	656	636	636
query30	289	153	160	153
query31	947	756	781	756
query32	98	61	55	55
query33	656	313	308	308
query34	922	488	498	488
query35	692	625	599	599
query36	1113	1005	983	983
query37	142	79	79	79
query38	2860	2795	2795	2795
query39	841	825	786	786
query40	194	119	120	119
query41	45	44	43	43
query42	112	96	97	96
query43	518	462	463	462
query44	1068	722	728	722
query45	190	159	159	159
query46	1074	747	689	689
query47	1838	1794	1774	1774
query48	360	295	291	291
query49	852	414	419	414
query50	785	386	380	380
query51	6938	6707	6849	6707
query52	106	93	93	93
query53	354	290	287	287
query54	861	440	447	440
query55	84	74	74	74
query56	286	260	274	260
query57	1140	1050	1052	1050
query58	264	245	268	245
query59	3013	2624	2636	2624
query60	298	294	273	273
query61	96	91	92	91
query62	769	668	662	662
query63	306	284	290	284
query64	9172	2252	1668	1668
query65	3178	3102	3111	3102
query66	727	323	329	323
query67	15569	14963	15010	14963
query68	5471	538	535	535
query69	616	480	373	373
query70	1211	1179	1146	1146
query71	425	282	282	282
query72	7922	5464	5668	5464
query73	785	319	324	319
query74	6105	5701	5752	5701
query75	3354	2725	2696	2696
query76	3175	955	951	951
query77	642	300	312	300
query78	11182	9194	9018	9018
query79	4800	523	531	523
query80	2308	467	462	462
query81	600	223	224	223
query82	770	137	132	132
query83	275	176	173	173
query84	281	88	87	87
query85	1167	316	289	289
query86	465	292	323	292
query87	3340	3113	3108	3108
query88	4305	2385	2370	2370
query89	480	403	384	384
query90	2034	192	192	192
query91	130	99	97	97
query92	63	55	48	48
query93	5551	507	500	500
query94	1426	208	206	206
query95	467	316	324	316
query96	623	272	275	272
query97	3185	3018	3034	3018
query98	224	197	191	191
query99	1557	1249	1290	1249
Total cold run time: 286332 ms
Total hot run time: 173762 ms

doris-robot avatar Jul 17 '24 04:07 doris-robot

ClickBench: Total hot run time: 30.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b156df0619f6ea95364534f7e25482e0262409e7, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.09	0.09
query5	0.50	0.50	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.48	0.49
query10	0.54	0.55	0.54
query11	0.16	0.11	0.12
query12	0.14	0.13	0.12
query13	0.59	0.59	0.58
query14	0.76	0.79	0.78
query15	0.85	0.81	0.83
query16	0.37	0.37	0.36
query17	1.00	1.02	1.03
query18	0.22	0.21	0.22
query19	1.89	1.72	1.73
query20	0.02	0.01	0.01
query21	15.42	0.74	0.66
query22	4.41	7.15	1.93
query23	18.31	1.38	1.21
query24	2.05	0.22	0.22
query25	0.14	0.08	0.08
query26	0.30	0.21	0.21
query27	0.46	0.24	0.23
query28	13.36	1.02	1.02
query29	12.61	3.31	3.28
query30	0.25	0.07	0.06
query31	2.84	0.39	0.40
query32	3.27	0.47	0.48
query33	2.84	2.96	2.91
query34	17.11	4.32	4.44
query35	4.45	4.40	4.45
query36	0.66	0.49	0.47
query37	0.18	0.16	0.16
query38	0.15	0.16	0.14
query39	0.04	0.03	0.04
query40	0.15	0.13	0.14
query41	0.09	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.01 s
Total hot run time: 30.63 s

doris-robot avatar Jul 17 '24 04:07 doris-robot

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In build.sh line 289:
    local submodule_path=$1
    ^---------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 290:
    local submodule_name=$2
    ^---------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 291:
    local archive_url=$3
    ^------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 293:
    set +e
    ^----^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 294:
    cd "${DORIS_HOME}"
    ^----------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 295:
    echo "Update ${submodule_name} submodule ..."
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 296:
    git submodule update --init --recursive "${submodule_path}"
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 297:
    exit_code=$?
    ^----------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 298:
    if [[ "${exit_code}" -eq 0 ]]; then
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
       ^------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 299:
        cd "${submodule_path}"
        ^--------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 300:
        submodule_commit_id=$(git rev-parse HEAD)
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                              ^----------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 301:
        cd -
        ^--^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 302:
        expect_submodule_commit_id=$(git ls-tree HEAD "${submodule_path}" | awk '{print $3}')
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                                     ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 303:
        echo "Current commit ID of ${submodule_name} submodule: ${submodule_commit_id}, expected is ${expect_submodule_commit_id}"
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 305:
    set -e
    ^----^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 306:
    if [[ "${exit_code}" -ne 0 ]]; then
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
       ^------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 307:
        set +e
        ^----^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 309:
        submodule_commit=$(git ls-tree HEAD "${submodule_path}" | awk '{print $3}')
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                           ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 310:
        exit_code=$?
        ^----------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 311:
        if [[ "${exit_code}" = "0" ]]; then
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
           ^------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 312:
            commit_specific_url=$(echo "${archive_url}" | sed "s/refs\/heads/${submodule_commit}/")
            ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                                  ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 314:
            commit_specific_url="${archive_url}"
            ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 316:
        set -e
        ^----^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 317:
        echo "Update ${submodule_name} submodule failed, start to download and extract ${commit_specific_url}"
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 319:
        mkdir -p "${DORIS_HOME}/${submodule_path}"
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 320:
        curl -L "${commit_specific_url}" | tar -xz -C "${DORIS_HOME}/${submodule_path}" --strip-components=1
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

For more information:
  https://www.shellcheck.net/wiki/SC2317 -- Command appears to be unreachable...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors
'shfmt ' found no issues.

github-actions[bot] avatar Jul 18 '24 07:07 github-actions[bot]

run buildall

wyxxxcat avatar Jul 23 '24 01:07 wyxxxcat

TPC-H: Total hot run time: 39642 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 507e6da57fde94e33b4c07c3b45993378d5ba2a8, data reload: false

------ Round 1 ----------------------------------
q1	17655	4283	4255	4255
q2	2013	191	182	182
q3	10449	1237	1083	1083
q4	10182	767	762	762
q5	7528	2677	2656	2656
q6	221	139	140	139
q7	949	590	604	590
q8	9221	2063	2087	2063
q9	8886	6552	6485	6485
q10	8783	3778	3804	3778
q11	474	234	242	234
q12	455	227	232	227
q13	17769	2993	2969	2969
q14	275	233	246	233
q15	530	482	492	482
q16	518	386	374	374
q17	971	629	726	629
q18	8130	7431	7540	7431
q19	5217	1458	1399	1399
q20	674	321	322	321
q21	4850	3075	3280	3075
q22	330	275	289	275
Total cold run time: 116080 ms
Total hot run time: 39642 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4378	4259	4222	4222
q2	368	266	272	266
q3	2959	2812	2904	2812
q4	2020	1797	1661	1661
q5	5682	5537	5547	5537
q6	233	134	134	134
q7	2165	1835	1842	1835
q8	3317	3502	3426	3426
q9	8811	8814	8942	8814
q10	4154	3935	3807	3807
q11	636	486	509	486
q12	816	626	642	626
q13	17001	3163	3195	3163
q14	324	294	290	290
q15	544	487	491	487
q16	488	440	435	435
q17	1828	1531	1510	1510
q18	8074	7931	7783	7783
q19	1742	1530	1564	1530
q20	2862	1901	1844	1844
q21	5118	5049	4848	4848
q22	586	500	534	500
Total cold run time: 74106 ms
Total hot run time: 56016 ms

doris-robot avatar Jul 23 '24 01:07 doris-robot

TPC-DS: Total hot run time: 173534 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 507e6da57fde94e33b4c07c3b45993378d5ba2a8, data reload: false

query1	927	375	373	373
query2	6444	1935	1768	1768
query3	6639	224	222	222
query4	24455	17780	17534	17534
query5	3700	478	496	478
query6	279	188	192	188
query7	4480	282	284	282
query8	243	198	197	197
query9	7905	2420	2387	2387
query10	435	277	278	277
query11	10510	9997	10039	9997
query12	123	95	88	88
query13	1526	387	362	362
query14	9903	7869	7878	7869
query15	226	165	164	164
query16	7653	446	484	446
query17	1126	538	539	538
query18	1809	277	270	270
query19	187	149	148	148
query20	86	83	79	79
query21	204	125	123	123
query22	4369	4024	3974	3974
query23	33985	33791	33372	33372
query24	10848	2899	2977	2899
query25	613	411	383	383
query26	719	157	153	153
query27	2252	267	284	267
query28	6185	2069	2085	2069
query29	930	648	675	648
query30	252	166	152	152
query31	984	796	766	766
query32	96	57	59	57
query33	646	356	346	346
query34	913	512	521	512
query35	884	767	768	767
query36	1149	1000	944	944
query37	151	82	83	82
query38	2920	2904	2821	2821
query39	922	879	861	861
query40	213	122	119	119
query41	44	44	44	44
query42	125	107	103	103
query43	503	450	468	450
query44	1075	721	725	721
query45	196	165	156	156
query46	1083	731	699	699
query47	1850	1751	1731	1731
query48	364	290	286	286
query49	827	429	411	411
query50	786	385	392	385
query51	6826	6675	6696	6675
query52	100	99	91	91
query53	360	285	291	285
query54	784	465	443	443
query55	84	74	74	74
query56	290	263	268	263
query57	1112	1061	1047	1047
query58	256	254	260	254
query59	2808	2696	2620	2620
query60	299	277	279	277
query61	97	94	94	94
query62	791	653	661	653
query63	317	289	290	289
query64	9147	2239	1703	1703
query65	3153	3129	3109	3109
query66	754	322	327	322
query67	15431	14772	14981	14772
query68	4537	519	527	519
query69	506	353	334	334
query70	1134	1138	1156	1138
query71	374	287	288	287
query72	7249	5341	5518	5341
query73	743	322	320	320
query74	6166	5694	5563	5563
query75	3707	2694	2695	2694
query76	2316	984	942	942
query77	593	318	310	310
query78	9545	9016	8892	8892
query79	2427	534	522	522
query80	2680	492	485	485
query81	615	223	221	221
query82	865	145	136	136
query83	320	178	175	175
query84	272	87	96	87
query85	1970	325	375	325
query86	489	348	324	324
query87	3280	3094	3076	3076
query88	3995	2353	2354	2353
query89	486	377	391	377
query90	1755	194	194	194
query91	133	108	103	103
query92	63	50	51	50
query93	3126	505	492	492
query94	1167	297	289	289
query95	419	326	318	318
query96	608	271	272	271
query97	3207	3004	3031	3004
query98	227	204	192	192
query99	1527	1282	1261	1261
Total cold run time: 273854 ms
Total hot run time: 173534 ms

doris-robot avatar Jul 23 '24 02:07 doris-robot

run buildall

wyxxxcat avatar Jul 23 '24 02:07 wyxxxcat

TPC-H: Total hot run time: 39732 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

------ Round 1 ----------------------------------
q1	17665	4330	4339	4330
q2	2013	192	197	192
q3	10435	1169	1027	1027
q4	10199	879	814	814
q5	7523	2715	2626	2626
q6	219	134	137	134
q7	947	595	604	595
q8	9220	2057	2085	2057
q9	8890	6575	6511	6511
q10	8903	3760	3750	3750
q11	469	233	237	233
q12	409	230	232	230
q13	17780	2992	2993	2992
q14	286	236	245	236
q15	535	502	492	492
q16	486	388	379	379
q17	972	632	646	632
q18	8109	7422	7583	7422
q19	7895	1432	1405	1405
q20	671	316	318	316
q21	4853	3076	3188	3076
q22	344	283	284	283
Total cold run time: 118823 ms
Total hot run time: 39732 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4390	4281	4267	4267
q2	369	265	269	265
q3	3035	2945	2914	2914
q4	1944	1692	1830	1692
q5	5617	5543	5510	5510
q6	222	130	129	129
q7	2232	1890	1867	1867
q8	3278	3452	3411	3411
q9	8805	8914	8824	8824
q10	4097	3689	3888	3689
q11	595	490	502	490
q12	820	656	622	622
q13	16188	3193	3203	3193
q14	311	293	298	293
q15	516	496	472	472
q16	506	443	438	438
q17	1793	1531	1497	1497
q18	8043	8033	7837	7837
q19	1741	1729	1554	1554
q20	2139	1853	1879	1853
q21	5179	4954	4863	4863
q22	564	506	493	493
Total cold run time: 72384 ms
Total hot run time: 56173 ms

doris-robot avatar Jul 23 '24 02:07 doris-robot

TPC-DS: Total hot run time: 174275 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

query1	914	387	364	364
query2	6426	1979	1901	1901
query3	6640	205	227	205
query4	28348	17474	17525	17474
query5	3620	476	474	474
query6	265	182	185	182
query7	4580	297	288	288
query8	253	197	191	191
query9	8501	2412	2394	2394
query10	433	294	301	294
query11	10757	10175	10248	10175
query12	114	86	78	78
query13	1630	359	352	352
query14	10159	7514	7552	7514
query15	222	161	168	161
query16	7756	453	450	450
query17	1548	567	528	528
query18	1961	273	272	272
query19	190	154	154	154
query20	89	82	89	82
query21	214	132	120	120
query22	4561	4296	4248	4248
query23	34108	33792	33668	33668
query24	10699	2902	2873	2873
query25	593	395	410	395
query26	705	155	150	150
query27	2287	278	269	269
query28	5874	2032	2035	2032
query29	890	656	621	621
query30	271	161	160	160
query31	985	748	754	748
query32	97	53	55	53
query33	681	333	330	330
query34	895	488	500	488
query35	881	726	749	726
query36	1179	968	986	968
query37	160	89	84	84
query38	2970	2883	2866	2866
query39	886	850	857	850
query40	205	126	124	124
query41	48	44	44	44
query42	122	109	101	101
query43	516	473	472	472
query44	1070	719	737	719
query45	203	161	163	161
query46	1107	719	732	719
query47	1868	1770	1775	1770
query48	359	291	291	291
query49	839	406	401	401
query50	776	384	387	384
query51	6814	6794	6740	6740
query52	102	92	91	91
query53	356	296	290	290
query54	888	450	440	440
query55	77	75	73	73
query56	288	261	274	261
query57	1156	1053	1115	1053
query58	253	245	240	240
query59	2881	2615	2569	2569
query60	297	298	277	277
query61	95	94	93	93
query62	826	646	657	646
query63	314	281	290	281
query64	9147	2208	1727	1727
query65	3148	3086	3091	3086
query66	737	328	326	326
query67	15579	14935	14827	14827
query68	6178	528	529	528
query69	689	448	379	379
query70	1161	1132	1065	1065
query71	479	283	276	276
query72	7788	5819	5619	5619
query73	822	317	319	317
query74	6118	5768	5720	5720
query75	3882	2667	2698	2667
query76	3899	1038	883	883
query77	695	318	307	307
query78	9845	8972	9006	8972
query79	2368	509	524	509
query80	1392	479	481	479
query81	555	219	221	219
query82	754	134	131	131
query83	231	167	170	167
query84	273	88	86	86
query85	1210	379	298	298
query86	404	310	318	310
query87	3280	3097	3115	3097
query88	3533	2352	2348	2348
query89	477	375	371	371
query90	1768	189	193	189
query91	131	100	102	100
query92	57	50	50	50
query93	2326	500	498	498
query94	948	274	299	274
query95	408	314	304	304
query96	589	266	273	266
query97	3231	3069	3018	3018
query98	228	208	199	199
query99	1564	1241	1280	1241
Total cold run time: 280696 ms
Total hot run time: 174275 ms

doris-robot avatar Jul 23 '24 02:07 doris-robot

ClickBench: Total hot run time: 30.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.22	0.04	0.05
query4	1.68	0.09	0.08
query5	0.51	0.48	0.48
query6	1.12	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.49
query10	0.55	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.58	0.57
query14	0.75	0.79	0.79
query15	0.85	0.81	0.82
query16	0.36	0.36	0.38
query17	0.97	0.94	1.01
query18	0.22	0.23	0.21
query19	1.87	1.69	1.73
query20	0.01	0.01	0.01
query21	15.40	0.76	0.64
query22	4.16	6.72	1.91
query23	18.31	1.43	1.25
query24	2.14	0.24	0.23
query25	0.16	0.09	0.08
query26	0.29	0.20	0.20
query27	0.46	0.23	0.23
query28	13.25	1.02	1.00
query29	12.59	3.30	3.32
query30	0.25	0.06	0.05
query31	2.86	0.38	0.39
query32	3.31	0.47	0.47
query33	2.88	2.90	2.92
query34	16.91	4.34	4.34
query35	4.45	4.41	4.43
query36	0.66	0.48	0.50
query37	0.19	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.12
query41	0.09	0.05	0.06
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.54 s
Total hot run time: 30.54 s

doris-robot avatar Jul 23 '24 02:07 doris-robot

run buildall

wyxxxcat avatar Jul 23 '24 14:07 wyxxxcat

TPC-H: Total hot run time: 40468 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

------ Round 1 ----------------------------------
q1	18149	4469	4371	4371
q2	2729	204	197	197
q3	10703	1190	1152	1152
q4	10316	878	800	800
q5	7918	2716	2715	2715
q6	229	140	140	140
q7	981	628	621	621
q8	9257	2096	2123	2096
q9	8795	6596	6549	6549
q10	8745	3821	3834	3821
q11	448	238	245	238
q12	399	237	221	221
q13	18786	2998	3010	2998
q14	278	231	236	231
q15	531	473	491	473
q16	510	388	380	380
q17	962	713	716	713
q18	8055	7640	7508	7508
q19	5732	1434	1380	1380
q20	669	325	344	325
q21	5005	3264	3251	3251
q22	356	288	291	288
Total cold run time: 119553 ms
Total hot run time: 40468 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4390	4255	4185	4185
q2	382	274	265	265
q3	3020	2724	2729	2724
q4	1830	1595	1602	1595
q5	5313	5332	5357	5332
q6	219	131	133	131
q7	2161	1776	1732	1732
q8	3203	3385	3324	3324
q9	8480	8427	8425	8425
q10	3912	3712	3688	3688
q11	594	498	491	491
q12	803	618	613	613
q13	16403	3004	3069	3004
q14	304	267	278	267
q15	528	486	478	478
q16	465	426	416	416
q17	1790	1456	1448	1448
q18	7661	7629	7471	7471
q19	1690	1590	1591	1590
q20	2010	1772	1791	1772
q21	4948	4842	4851	4842
q22	577	501	517	501
Total cold run time: 70683 ms
Total hot run time: 54294 ms

doris-robot avatar Jul 23 '24 14:07 doris-robot

TPC-DS: Total hot run time: 173639 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

query1	927	402	369	369
query2	6440	1919	1814	1814
query3	6653	208	217	208
query4	24074	17521	17251	17251
query5	4173	484	470	470
query6	277	192	159	159
query7	4591	309	310	309
query8	251	217	201	201
query9	8555	2451	2450	2450
query10	446	281	277	277
query11	11432	10096	10105	10096
query12	137	87	89	87
query13	1656	388	386	386
query14	10465	7527	7725	7527
query15	227	166	170	166
query16	7809	441	476	441
query17	1581	559	525	525
query18	1902	280	274	274
query19	199	149	156	149
query20	99	81	81	81
query21	222	139	130	130
query22	4340	4014	4008	4008
query23	33853	33340	33239	33239
query24	11236	2851	2918	2851
query25	635	374	373	373
query26	1560	152	154	152
query27	2965	271	276	271
query28	7515	2037	2023	2023
query29	1005	645	610	610
query30	285	153	148	148
query31	941	749	743	743
query32	99	58	55	55
query33	775	335	335	335
query34	901	482	493	482
query35	910	751	761	751
query36	1095	965	946	946
query37	147	78	80	78
query38	2812	2786	2796	2786
query39	876	796	828	796
query40	268	125	119	119
query41	48	73	43	43
query42	125	98	102	98
query43	493	467	459	459
query44	1193	734	723	723
query45	189	161	163	161
query46	1103	752	732	732
query47	1903	1776	1784	1776
query48	377	294	295	294
query49	1165	413	419	413
query50	782	399	394	394
query51	6839	6789	6712	6712
query52	115	97	97	97
query53	362	295	301	295
query54	889	453	466	453
query55	78	73	76	73
query56	287	268	275	268
query57	1177	1053	1068	1053
query58	254	239	253	239
query59	2809	2648	2675	2648
query60	299	284	310	284
query61	97	94	96	94
query62	854	659	670	659
query63	321	311	289	289
query64	10393	2220	1673	1673
query65	3189	3126	3095	3095
query66	1332	330	335	330
query67	15374	14857	15105	14857
query68	4565	565	572	565
query69	475	371	392	371
query70	1174	1171	1176	1171
query71	417	285	290	285
query72	7134	5696	5544	5544
query73	758	328	331	328
query74	6152	5739	5624	5624
query75	3475	2712	2695	2695
query76	2809	1004	947	947
query77	464	310	301	301
query78	9876	9234	9086	9086
query79	2310	542	533	533
query80	2308	498	489	489
query81	596	224	229	224
query82	787	143	134	134
query83	304	171	172	171
query84	263	86	99	86
query85	2169	329	301	301
query86	483	320	289	289
query87	3243	3069	3108	3069
query88	4266	2396	2406	2396
query89	494	395	404	395
query90	1857	201	199	199
query91	146	116	117	116
query92	142	51	51	51
query93	2294	521	514	514
query94	1268	299	304	299
query95	405	320	326	320
query96	609	277	272	272
query97	3183	3080	3046	3046
query98	217	199	201	199
query99	1509	1283	1271	1271
Total cold run time: 282533 ms
Total hot run time: 173639 ms

doris-robot avatar Jul 23 '24 15:07 doris-robot

ClickBench: Total hot run time: 30.83 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 06a0f2418b23710f200d6af7fdf25d6c59646ae3, data reload: false

query1	0.05	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.09	0.08
query5	0.50	0.48	0.48
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.47	0.48
query10	0.55	0.55	0.54
query11	0.16	0.11	0.11
query12	0.15	0.13	0.13
query13	0.59	0.58	0.59
query14	0.77	0.78	0.77
query15	0.84	0.81	0.82
query16	0.37	0.36	0.35
query17	1.00	0.98	0.98
query18	0.23	0.22	0.22
query19	1.78	1.72	1.67
query20	0.02	0.01	0.01
query21	15.39	0.76	0.66
query22	4.20	7.28	1.93
query23	18.25	1.40	1.28
query24	2.07	0.24	0.23
query25	0.16	0.09	0.09
query26	0.31	0.22	0.22
query27	0.46	0.23	0.23
query28	13.25	1.02	1.00
query29	12.59	3.38	3.36
query30	0.25	0.05	0.06
query31	2.89	0.40	0.38
query32	3.26	0.47	0.48
query33	2.90	2.93	2.92
query34	16.81	4.40	4.41
query35	4.49	4.43	4.49
query36	0.65	0.46	0.48
query37	0.19	0.17	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.14	0.12	0.13
query41	0.10	0.05	0.04
query42	0.06	0.06	0.06
query43	0.06	0.04	0.04
Total cold run time: 109.43 s
Total hot run time: 30.83 s

doris-robot avatar Jul 23 '24 15:07 doris-robot

run buildall

wyxxxcat avatar Jul 29 '24 04:07 wyxxxcat

run buildall

wyxxxcat avatar Jul 29 '24 13:07 wyxxxcat

run buildall

wyxxxcat avatar Jul 30 '24 02:07 wyxxxcat