doris icon indicating copy to clipboard operation
doris copied to clipboard

[feature](expr) Add length limit to the debug string of expr

Open Mryange opened this issue 10 months ago • 16 comments

Proposed changes

The debug string may be too long, so we'll truncate it here.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

Mryange avatar Apr 11 '24 13:04 Mryange

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar Apr 11 '24 13:04 doris-robot

run buildall

Mryange avatar Apr 11 '24 13:04 Mryange

TeamCity be ut coverage result: Function Coverage: 35.59% (8904/25016) Line Coverage: 27.31% (73107/267706) Region Coverage: 26.44% (37814/143029) Branch Coverage: 23.19% (19270/83088) Coverage Report: http://coverage.selectdb-in.cc/coverage/363bdda0f758236f2c88773203d1d3dd01333ccb_363bdda0f758236f2c88773203d1d3dd01333ccb/report/index.html

doris-robot avatar Apr 11 '24 15:04 doris-robot

run buildall

Mryange avatar Apr 17 '24 08:04 Mryange

TPC-H: Total hot run time: 38756 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

------ Round 1 ----------------------------------
q1	17631	4633	4267	4267
q2	2012	193	186	186
q3	10703	1130	1214	1130
q4	10259	926	712	712
q5	7815	2794	2717	2717
q6	230	135	131	131
q7	969	550	546	546
q8	9483	2141	2081	2081
q9	7422	6747	6720	6720
q10	9174	3507	3531	3507
q11	463	230	236	230
q12	435	217	209	209
q13	17950	2970	2935	2935
q14	278	224	237	224
q15	532	489	487	487
q16	535	380	374	374
q17	972	751	730	730
q18	7486	6830	6767	6767
q19	4598	1524	1519	1519
q20	659	319	296	296
q21	3640	2681	2762	2681
q22	358	307	308	307
Total cold run time: 113604 ms
Total hot run time: 38756 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4333	4182	4207	4182
q2	380	264	272	264
q3	3016	2717	2781	2717
q4	1846	1600	1576	1576
q5	5370	5354	5326	5326
q6	213	123	121	121
q7	1792	1435	1404	1404
q8	3218	3351	3368	3351
q9	8599	8592	8611	8592
q10	3935	3733	3633	3633
q11	574	479	477	477
q12	739	606	577	577
q13	16331	2967	2912	2912
q14	305	267	271	267
q15	511	475	477	475
q16	472	416	426	416
q17	1783	1469	1464	1464
q18	7647	7521	7498	7498
q19	1638	1532	1534	1532
q20	1952	1770	1768	1768
q21	4918	4762	4730	4730
q22	541	445	477	445
Total cold run time: 70113 ms
Total hot run time: 53727 ms

doris-robot avatar Apr 17 '24 09:04 doris-robot

TPC-DS: Total hot run time: 184242 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

query1	905	377	366	366
query2	7184	2443	2369	2369
query3	6648	200	202	200
query4	24056	21349	21341	21341
query5	4158	402	409	402
query6	262	176	173	173
query7	4575	285	286	285
query8	227	173	173	173
query9	8517	2322	2306	2306
query10	586	240	265	240
query11	15709	14228	14194	14194
query12	138	92	87	87
query13	1647	366	361	361
query14	9316	8008	7830	7830
query15	260	194	187	187
query16	8246	259	259	259
query17	1994	592	568	568
query18	2116	285	288	285
query19	339	155	160	155
query20	95	85	88	85
query21	204	133	126	126
query22	5013	4822	4875	4822
query23	33871	33069	32976	32976
query24	11193	2960	2940	2940
query25	622	359	361	359
query26	1162	149	146	146
query27	2827	311	302	302
query28	7453	1979	1972	1972
query29	856	602	584	584
query30	309	171	177	171
query31	971	726	726	726
query32	94	51	52	51
query33	741	240	243	240
query34	1016	465	467	465
query35	867	704	664	664
query36	1075	922	869	869
query37	133	71	68	68
query38	3382	3201	3221	3201
query39	1562	1534	1513	1513
query40	204	123	124	123
query41	47	43	42	42
query42	103	92	94	92
query43	560	539	561	539
query44	1173	714	719	714
query45	273	259	279	259
query46	1071	703	736	703
query47	1941	1846	1859	1846
query48	362	289	294	289
query49	988	372	362	362
query50	744	372	362	362
query51	6610	6568	6602	6568
query52	110	86	89	86
query53	350	289	274	274
query54	298	227	221	221
query55	73	72	69	69
query56	236	224	222	222
query57	1195	1124	1111	1111
query58	218	191	191	191
query59	3534	3365	3133	3133
query60	269	226	239	226
query61	90	87	86	86
query62	672	447	440	440
query63	307	271	268	268
query64	5222	3790	3844	3790
query65	3066	3031	2997	2997
query66	1428	343	332	332
query67	15491	15117	15026	15026
query68	5301	521	530	521
query69	474	305	292	292
query70	1226	1164	1207	1164
query71	1383	1266	1263	1263
query72	6683	2617	2426	2426
query73	709	311	310	310
query74	6857	6436	6413	6413
query75	3346	2661	2623	2623
query76	2864	943	954	943
query77	402	272	252	252
query78	10925	10071	10176	10071
query79	5566	541	502	502
query80	1819	432	424	424
query81	547	246	243	243
query82	1284	100	92	92
query83	318	169	167	167
query84	275	85	82	82
query85	1998	264	258	258
query86	530	299	290	290
query87	3510	3246	3312	3246
query88	5164	2298	2291	2291
query89	487	364	365	364
query90	1975	176	180	176
query91	124	97	100	97
query92	61	48	48	48
query93	5583	514	490	490
query94	1211	177	172	172
query95	385	300	291	291
query96	594	258	257	257
query97	3111	2969	2953	2953
query98	234	216	216	216
query99	1207	893	873	873
Total cold run time: 293117 ms
Total hot run time: 184242 ms

doris-robot avatar Apr 17 '24 09:04 doris-robot

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

query1	0.03	0.02	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.50	0.48	0.50
query6	1.48	0.73	0.71
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.55	0.49	0.50
query10	0.56	0.56	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.58
query14	0.76	0.78	0.77
query15	0.83	0.80	0.80
query16	0.37	0.36	0.36
query17	1.01	0.98	0.99
query18	0.20	0.24	0.22
query19	1.80	1.69	1.74
query20	0.02	0.01	0.01
query21	15.43	0.65	0.65
query22	3.84	7.83	2.12
query23	18.34	1.42	1.24
query24	1.57	0.26	0.26
query25	0.13	0.08	0.08
query26	0.27	0.15	0.16
query27	0.07	0.08	0.07
query28	13.33	0.99	0.99
query29	12.67	3.24	3.24
query30	0.27	0.07	0.05
query31	2.87	0.37	0.37
query32	3.28	0.46	0.46
query33	2.74	2.83	2.82
query34	17.20	4.38	4.40
query35	4.49	4.47	4.46
query36	0.65	0.46	0.47
query37	0.17	0.15	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.19	0.17	0.14
query41	0.09	0.05	0.04
query42	0.06	0.05	0.04
query43	0.03	0.03	0.04
Total cold run time: 108.99 s
Total hot run time: 30.46 s

doris-robot avatar Apr 17 '24 09:04 doris-robot

TeamCity be ut coverage result: Function Coverage: 35.45% (8907/25128) Line Coverage: 27.14% (73149/269552) Region Coverage: 26.27% (37827/143972) Branch Coverage: 23.07% (19274/83546) Coverage Report: http://coverage.selectdb-in.cc/coverage/83c7e0f6928d11071a9e52b9375c662535cd86dd_83c7e0f6928d11071a9e52b9375c662535cd86dd/report/index.html

doris-robot avatar Apr 17 '24 09:04 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

doris-robot avatar Apr 17 '24 09:04 doris-robot

run buildall

Mryange avatar Apr 17 '24 15:04 Mryange

TPC-H: Total hot run time: 38675 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

------ Round 1 ----------------------------------
q1	18820	5003	4259	4259
q2	2010	192	196	192
q3	10541	1154	1166	1154
q4	10263	787	803	787
q5	7529	2769	2726	2726
q6	220	137	135	135
q7	1041	605	588	588
q8	9219	2114	2066	2066
q9	7237	6693	6577	6577
q10	8556	3544	3532	3532
q11	445	230	231	230
q12	397	214	215	214
q13	17780	2937	2954	2937
q14	267	241	235	235
q15	522	495	466	466
q16	519	385	373	373
q17	975	661	779	661
q18	7474	6911	6729	6729
q19	6439	1548	1545	1545
q20	683	311	302	302
q21	3533	2689	2663	2663
q22	358	308	304	304
Total cold run time: 114828 ms
Total hot run time: 38675 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4289	4253	4244	4244
q2	381	270	275	270
q3	2987	2741	2739	2739
q4	1890	1588	1544	1544
q5	5376	5361	5336	5336
q6	210	126	124	124
q7	2259	1820	1878	1820
q8	3255	3382	3361	3361
q9	8631	8642	8694	8642
q10	3920	3730	3702	3702
q11	588	469	468	468
q12	774	589	597	589
q13	17237	2941	2938	2938
q14	301	269	270	269
q15	515	480	463	463
q16	474	431	414	414
q17	1793	1493	1469	1469
q18	7681	7452	7488	7452
q19	1640	1505	1586	1505
q20	1955	1750	1729	1729
q21	9487	4736	4741	4736
q22	538	466	461	461
Total cold run time: 76181 ms
Total hot run time: 54275 ms

doris-robot avatar Apr 17 '24 15:04 doris-robot

TPC-DS: Total hot run time: 184582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

query1	906	370	376	370
query2	6479	2498	2451	2451
query3	6653	204	205	204
query4	25482	21604	21382	21382
query5	4196	407	425	407
query6	273	172	176	172
query7	4591	288	281	281
query8	234	192	177	177
query9	8596	2295	2284	2284
query10	588	243	256	243
query11	14942	14212	14278	14212
query12	138	95	84	84
query13	1645	371	375	371
query14	9830	7810	7221	7221
query15	257	200	198	198
query16	8162	266	263	263
query17	1924	597	558	558
query18	2111	288	276	276
query19	271	156	160	156
query20	93	87	87	87
query21	203	127	125	125
query22	5076	4808	4881	4808
query23	33872	33236	33406	33236
query24	12126	2900	2927	2900
query25	669	381	392	381
query26	1763	162	157	157
query27	3117	314	319	314
query28	7646	2020	1995	1995
query29	1034	625	612	612
query30	319	173	177	173
query31	971	748	730	730
query32	94	58	53	53
query33	775	258	245	245
query34	1064	483	497	483
query35	823	700	682	682
query36	1046	877	911	877
query37	275	74	74	74
query38	3293	3225	3186	3186
query39	1598	1522	1571	1522
query40	276	127	131	127
query41	51	114	43	43
query42	104	97	94	94
query43	539	553	551	551
query44	1248	727	741	727
query45	284	265	254	254
query46	1061	724	733	724
query47	1932	1853	1829	1829
query48	361	304	300	300
query49	1184	369	369	369
query50	756	388	391	388
query51	6723	6652	6661	6652
query52	97	96	89	89
query53	348	283	291	283
query54	327	233	226	226
query55	79	71	70	70
query56	240	222	218	218
query57	1259	1132	1133	1132
query58	217	195	199	195
query59	3488	3234	3023	3023
query60	260	225	243	225
query61	108	86	94	86
query62	651	433	433	433
query63	306	282	284	282
query64	6282	3891	3805	3805
query65	3143	3046	3114	3046
query66	1370	340	337	337
query67	15470	15056	14933	14933
query68	7879	552	555	552
query69	541	307	303	303
query70	1298	1170	1187	1170
query71	1476	1295	1262	1262
query72	6379	2627	2417	2417
query73	725	327	323	323
query74	6899	6394	6511	6394
query75	4097	2663	2573	2573
query76	4660	985	997	985
query77	649	266	263	263
query78	10879	10266	10228	10228
query79	7612	523	526	523
query80	1034	434	436	434
query81	519	240	241	240
query82	238	91	102	91
query83	196	189	172	172
query84	264	86	81	81
query85	1012	270	253	253
query86	351	320	295	295
query87	3476	3279	3288	3279
query88	4993	2408	2412	2408
query89	510	378	364	364
query90	2035	183	183	183
query91	124	97	97	97
query92	57	48	47	47
query93	5928	526	510	510
query94	1281	181	178	178
query95	395	301	294	294
query96	611	268	268	268
query97	3144	2974	2937	2937
query98	237	214	216	214
query99	1076	870	853	853
Total cold run time: 301874 ms
Total hot run time: 184582 ms

doris-robot avatar Apr 17 '24 16:04 doris-robot

ClickBench: Total hot run time: 29.93 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.49	0.50	0.51
query6	1.45	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.55	0.49	0.50
query10	0.53	0.56	0.55
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.63	0.59	0.59
query14	0.75	0.77	0.76
query15	0.83	0.81	0.79
query16	0.36	0.39	0.37
query17	1.02	0.99	1.04
query18	0.20	0.24	0.24
query19	1.81	1.79	1.77
query20	0.01	0.01	0.01
query21	15.43	0.64	0.66
query22	4.82	7.38	1.35
query23	18.24	1.34	1.29
query24	1.86	0.22	0.22
query25	0.14	0.08	0.08
query26	0.26	0.16	0.16
query27	0.07	0.08	0.07
query28	13.32	0.99	0.99
query29	12.65	3.24	3.22
query30	0.26	0.06	0.06
query31	2.84	0.38	0.39
query32	3.27	0.47	0.46
query33	2.79	2.80	2.82
query34	16.94	4.46	4.42
query35	4.51	4.53	4.49
query36	0.65	0.46	0.46
query37	0.19	0.15	0.15
query38	0.16	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.03	0.03
Total cold run time: 109.85 s
Total hot run time: 29.93 s

doris-robot avatar Apr 17 '24 16:04 doris-robot

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 83c7e0f6928d11071a9e52b9375c662535cd86dd with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.5 seconds inserted 10000000 Rows, about 689K ops/s

doris-robot avatar Apr 17 '24 16:04 doris-robot

TeamCity be ut coverage result: Function Coverage: 35.44% (8905/25130) Line Coverage: 27.12% (73125/269621) Region Coverage: 26.25% (37803/144002) Branch Coverage: 23.05% (19263/83564) Coverage Report: http://coverage.selectdb-in.cc/coverage/83c7e0f6928d11071a9e52b9375c662535cd86dd_83c7e0f6928d11071a9e52b9375c662535cd86dd/report/index.html

doris-robot avatar Apr 17 '24 16:04 doris-robot

run buidlall

Mryange avatar Apr 18 '24 01:04 Mryange