doris icon indicating copy to clipboard operation
doris copied to clipboard

[chore](function) non_nullable function output nullmap if has null

Open Mryange opened this issue 2 weeks ago • 1 comments

What problem does this PR solve?

[INVALID_ARGUMENT]There's NULL value in column column which is illegal for non_nullable , null map: 0,1,0,

Release note

None

Check List (For Author)

  • Test

    • [ ] Regression test
    • [ ] Unit Test
    • [ ] Manual test (add detailed scripts or steps below)
    • [x] No need to test or manual test. Explain why:
      • [x] This is a refactor/code format and no logic has been changed.
      • [ ] Previous test can cover this change.
      • [ ] No code files have been changed.
      • [ ] Other reason
  • Behavior changed:

    • [x] No.
    • [ ] Yes.
  • Does this need documentation?

    • [x] No.
    • [ ] Yes.

Check List (For Reviewer who merge this PR)

  • [ ] Confirm the release note
  • [ ] Confirm test cases
  • [ ] Confirm document
  • [ ] Add branch pick label

Mryange avatar Dec 11 '25 06:12 Mryange

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Thearas avatar Dec 11 '25 06:12 Thearas

run buildall

Mryange avatar Dec 15 '25 04:12 Mryange

TPC-H: Total hot run time: 35041 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

------ Round 1 ----------------------------------
q1	17688	4360	4107	4107
q2	2070	367	236	236
q3	10122	1360	757	757
q4	10229	852	310	310
q5	7481	2144	1943	1943
q6	192	171	135	135
q7	1007	870	731	731
q8	9383	1458	1171	1171
q9	7084	5437	5338	5338
q10	6872	2386	1971	1971
q11	536	324	296	296
q12	714	729	565	565
q13	17786	3673	3018	3018
q14	288	291	288	288
q15	601	519	513	513
q16	693	672	637	637
q17	689	757	580	580
q18	7558	7195	7026	7026
q19	1129	973	595	595
q20	389	353	257	257
q21	4265	3952	3612	3612
q22	1034	1013	955	955
Total cold run time: 107810 ms
Total hot run time: 35041 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4128	4111	4115	4111
q2	322	396	301	301
q3	2139	2691	2291	2291
q4	1354	1784	1310	1310
q5	4247	4602	4782	4602
q6	245	187	139	139
q7	2114	1980	1832	1832
q8	2680	2564	2527	2527
q9	7504	7610	7507	7507
q10	3058	3222	2821	2821
q11	612	511	496	496
q12	723	823	686	686
q13	3581	3897	3361	3361
q14	291	326	302	302
q15	588	522	499	499
q16	668	734	632	632
q17	1189	1561	1467	1467
q18	8054	7673	7732	7673
q19	930	881	912	881
q20	1926	1970	1854	1854
q21	4668	4370	4170	4170
q22	1064	1020	974	974
Total cold run time: 52085 ms
Total hot run time: 50436 ms

doris-robot avatar Dec 15 '25 05:12 doris-robot

TPC-DS: Total hot run time: 178491 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

query5	4894	630	508	508
query6	350	238	222	222
query7	4230	468	288	288
query8	328	269	246	246
query9	8782	2591	2589	2589
query10	537	379	343	343
query11	15417	14817	14837	14817
query12	200	118	117	117
query13	1261	492	419	419
query14	6163	3233	2993	2993
query14_1	2874	2950	2866	2866
query15	217	204	183	183
query16	934	476	460	460
query17	1126	724	610	610
query18	2622	459	356	356
query19	238	229	219	219
query20	126	119	113	113
query21	230	146	120	120
query22	3874	3965	3887	3887
query23	16536	16174	15930	15930
query23_1	16070	16047	16017	16017
query24	7346	1661	1220	1220
query24_1	1264	1228	1259	1228
query25	606	503	454	454
query26	1274	271	174	174
query27	2740	472	317	317
query28	4456	2162	2157	2157
query29	793	541	440	440
query30	311	245	215	215
query31	828	712	631	631
query32	76	76	67	67
query33	526	342	280	280
query34	912	909	548	548
query35	775	854	743	743
query36	855	913	828	828
query37	129	92	76	76
query38	2877	2905	2800	2800
query39	760	783	717	717
query39_1	706	700	701	700
query40	221	136	120	120
query41	102	63	61	61
query42	111	104	103	103
query43	415	446	408	408
query44	1313	741	746	741
query45	195	184	182	182
query46	867	975	603	603
query47	1644	1719	1628	1628
query48	314	322	247	247
query49	619	433	359	359
query50	662	290	212	212
query51	3825	3861	3834	3834
query52	106	115	101	101
query53	333	349	295	295
query54	289	265	255	255
query55	82	76	73	73
query56	305	303	295	295
query57	1150	1132	1085	1085
query58	263	253	252	252
query59	2413	2504	2350	2350
query60	319	315	285	285
query61	166	161	165	161
query62	730	678	627	627
query63	342	293	298	293
query64	5047	1319	1007	1007
query65	3978	3943	3976	3943
query66	1426	447	318	318
query67	15046	14991	14730	14730
query68	8291	990	732	732
query69	514	347	319	319
query70	1039	948	950	948
query71	397	322	294	294
query72	6099	4937	4972	4937
query73	673	591	308	308
query74	8937	8733	8632	8632
query75	3172	3122	2749	2749
query76	3967	1139	760	760
query77	517	403	293	293
query78	9512	9703	8800	8800
query79	1865	839	628	628
query80	1163	652	548	548
query81	576	266	245	245
query82	409	132	105	105
query83	266	257	237	237
query84	266	127	107	107
query85	958	550	477	477
query86	427	304	314	304
query87	3050	3067	2903	2903
query88	3262	2262	2293	2262
query89	458	423	413	413
query90	2014	160	162	160
query91	173	168	154	154
query92	77	69	67	67
query93	1228	888	573	573
query94	520	294	287	287
query95	575	377	305	305
query96	589	456	206	206
query97	2270	2313	2200	2200
query98	211	191	194	191
query99	1256	1278	1209	1209
Total cold run time: 261073 ms
Total hot run time: 178491 ms

doris-robot avatar Dec 15 '25 05:12 doris-robot

ClickBench: Total hot run time: 27.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.04
query3	0.27	0.09	0.09
query4	1.61	0.11	0.10
query5	0.28	0.27	0.25
query6	1.17	0.64	0.63
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.58	0.51	0.50
query10	0.56	0.57	0.55
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.60
query14	0.99	1.01	0.98
query15	0.81	0.79	0.81
query16	0.40	0.40	0.40
query17	0.96	1.03	1.04
query18	0.22	0.21	0.20
query19	1.94	1.90	1.84
query20	0.01	0.01	0.02
query21	15.45	0.29	0.14
query22	4.82	0.05	0.05
query23	16.09	0.29	0.10
query24	0.95	0.24	0.58
query25	0.08	0.06	0.05
query26	0.14	0.14	0.14
query27	0.06	0.07	0.06
query28	3.97	1.22	1.02
query29	12.58	4.02	3.25
query30	0.29	0.14	0.11
query31	2.82	0.64	0.39
query32	3.24	0.56	0.47
query33	2.96	3.06	3.07
query34	16.74	5.15	4.47
query35	4.60	4.59	4.53
query36	0.66	0.50	0.49
query37	0.10	0.07	0.07
query38	0.07	0.05	0.04
query39	0.05	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 96.99 s
Total hot run time: 27.28 s

doris-robot avatar Dec 15 '25 05:12 doris-robot

BE UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 53.49% (18854/35246)
Line Coverage 39.24% (174462/444623)
Region Coverage 33.84% (135006/398906)
Branch Coverage 34.82% (58148/166990)

hello-stephen avatar Dec 15 '25 06:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 72.42% (25013/34540)
Line Coverage 59.18% (262776/444001)
Region Coverage 54.10% (218310/403539)
Branch Coverage 55.65% (93391/167805)

hello-stephen avatar Dec 15 '25 09:12 hello-stephen

run nonConcurrent

Mryange avatar Dec 17 '25 02:12 Mryange

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 73.33% (25327/34540)
Line Coverage 60.50% (268605/444001)
Region Coverage 56.03% (226093/403539)
Branch Coverage 57.37% (96269/167805)

hello-stephen avatar Dec 17 '25 04:12 hello-stephen

run buildall

Mryange avatar Dec 19 '25 06:12 Mryange

TPC-H: Total hot run time: 36497 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

------ Round 1 ----------------------------------
q1	17637	4277	4053	4053
q2	2037	371	247	247
q3	10168	1315	736	736
q4	10210	800	313	313
q5	7527	2124	1948	1948
q6	190	173	138	138
q7	992	867	707	707
q8	9378	1470	1168	1168
q9	7129	5344	5482	5344
q10	6861	2404	1965	1965
q11	544	332	301	301
q12	671	727	597	597
q13	17777	3679	3036	3036
q14	302	298	280	280
q15	582	519	510	510
q16	721	702	638	638
q17	705	829	595	595
q18	7623	7810	7749	7749
q19	1414	1030	645	645
q20	454	395	256	256
q21	4677	4274	4237	4237
q22	1166	1073	1034	1034
Total cold run time: 108765 ms
Total hot run time: 36497 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4413	4273	4410	4273
q2	387	419	336	336
q3	2312	2808	2541	2541
q4	1425	1858	1364	1364
q5	4585	4449	4645	4449
q6	211	170	130	130
q7	2076	1957	1839	1839
q8	2674	2542	2586	2542
q9	7564	7512	7438	7438
q10	3041	3246	2890	2890
q11	591	514	475	475
q12	695	712	577	577
q13	3279	3661	3016	3016
q14	274	281	326	281
q15	537	495	506	495
q16	631	648	580	580
q17	1127	1339	1396	1339
q18	7335	7054	7035	7035
q19	828	789	809	789
q20	1896	1968	1780	1780
q21	4618	4294	4092	4092
q22	1079	1019	980	980
Total cold run time: 51578 ms
Total hot run time: 49241 ms

doris-robot avatar Dec 19 '25 06:12 doris-robot

TPC-DS: Total hot run time: 178896 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

query5	4959	594	464	464
query6	348	226	209	209
query7	4236	470	271	271
query8	309	249	250	249
query9	8777	2523	2532	2523
query10	509	363	322	322
query11	15389	15146	14609	14609
query12	194	129	122	122
query13	1260	488	396	396
query14	6343	3023	2769	2769
query14_1	2626	2634	2709	2634
query15	213	240	186	186
query16	785	468	433	433
query17	1083	686	578	578
query18	2618	432	334	334
query19	233	232	201	201
query20	117	121	111	111
query21	219	139	113	113
query22	4019	3944	3936	3936
query23	16519	16125	15991	15991
query23_1	16002	16036	16034	16034
query24	7360	1664	1229	1229
query24_1	1255	1224	1248	1224
query25	557	471	429	429
query26	1254	280	166	166
query27	2755	465	313	313
query28	4494	2136	2136	2136
query29	817	552	437	437
query30	314	247	219	219
query31	850	690	611	611
query32	84	68	68	68
query33	557	338	292	292
query34	896	907	552	552
query35	784	841	748	748
query36	858	902	824	824
query37	122	88	83	83
query38	2875	2854	2828	2828
query39	758	739	717	717
query39_1	705	709	706	706
query40	226	151	126	126
query41	73	70	67	67
query42	113	108	108	108
query43	430	427	409	409
query44	1360	761	747	747
query45	195	195	195	195
query46	903	993	632	632
query47	1669	1679	1610	1610
query48	334	352	264	264
query49	645	445	370	370
query50	679	299	233	233
query51	3818	3838	3889	3838
query52	108	120	100	100
query53	340	359	302	302
query54	309	277	260	260
query55	79	85	77	77
query56	313	310	300	300
query57	1162	1181	1060	1060
query58	289	264	268	264
query59	2430	2493	2418	2418
query60	331	324	299	299
query61	193	187	190	187
query62	731	657	642	642
query63	334	298	304	298
query64	5070	1427	1116	1116
query65	4009	4006	3984	3984
query66	1433	447	335	335
query67	15416	15114	14885	14885
query68	7527	1031	732	732
query69	508	403	314	314
query70	1092	994	959	959
query71	378	303	287	287
query72	6035	4954	4900	4900
query73	650	572	316	316
query74	8886	8844	8650	8650
query75	3187	3139	2813	2813
query76	3839	1141	742	742
query77	522	405	288	288
query78	9458	9683	8905	8905
query79	1273	933	615	615
query80	699	662	544	544
query81	535	271	236	236
query82	202	131	103	103
query83	268	259	245	245
query84	267	126	106	106
query85	896	516	475	475
query86	384	292	276	276
query87	3062	3049	3002	3002
query88	3221	2297	2282	2282
query89	469	420	391	391
query90	2216	163	153	153
query91	172	167	144	144
query92	83	67	66	66
query93	1641	921	569	569
query94	497	291	275	275
query95	594	373	314	314
query96	602	461	212	212
query97	2259	2311	2236	2236
query98	215	193	188	188
query99	1290	1309	1208	1208
Total cold run time: 259840 ms
Total hot run time: 178896 ms

doris-robot avatar Dec 19 '25 06:12 doris-robot

ClickBench: Total hot run time: 27.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e95ee12b62d1280072253690904ee6fca8684c44, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.05
query3	0.27	0.09	0.09
query4	1.61	0.12	0.11
query5	0.26	0.25	0.26
query6	1.20	0.65	0.63
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.58	0.51	0.50
query10	0.56	0.54	0.56
query11	0.15	0.11	0.12
query12	0.15	0.12	0.13
query13	0.62	0.60	0.60
query14	0.99	0.99	0.98
query15	0.80	0.81	0.81
query16	0.41	0.39	0.40
query17	1.02	1.05	0.98
query18	0.24	0.22	0.21
query19	1.92	1.76	1.82
query20	0.02	0.01	0.01
query21	15.49	0.30	0.15
query22	4.71	0.05	0.05
query23	16.06	0.29	0.11
query24	0.94	0.66	0.18
query25	0.13	0.07	0.11
query26	0.13	0.13	0.12
query27	0.06	0.06	0.05
query28	3.94	1.24	1.04
query29	12.59	4.11	3.22
query30	0.30	0.14	0.11
query31	2.83	0.65	0.40
query32	3.24	0.55	0.46
query33	2.96	3.03	3.05
query34	17.00	5.15	4.49
query35	4.54	4.61	4.55
query36	0.66	0.50	0.49
query37	0.11	0.06	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.04
query40	0.17	0.15	0.14
query41	0.09	0.03	0.02
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 97.21 s
Total hot run time: 27.11 s

doris-robot avatar Dec 19 '25 06:12 doris-robot

BE UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 53.42% (18855/35296)
Line Coverage 39.25% (174795/445319)
Region Coverage 33.82% (135378/400246)
Branch Coverage 34.72% (58308/167928)

hello-stephen avatar Dec 19 '25 07:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 73.00% (25241/34578)
Line Coverage 59.79% (265735/444416)
Region Coverage 55.01% (222545/404542)
Branch Coverage 56.22% (94790/168601)

hello-stephen avatar Dec 19 '25 07:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 73.03% (25255/34582)
Line Coverage 59.86% (266138/444628)
Region Coverage 55.08% (222975/404793)
Branch Coverage 56.29% (94968/168700)

hello-stephen avatar Dec 19 '25 08:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 73.70% (25488/34582)
Line Coverage 60.93% (270921/444628)
Region Coverage 55.71% (225504/404793)
Branch Coverage 57.54% (97065/168700)

hello-stephen avatar Dec 19 '25 09:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 73.70% (25488/34582)
Line Coverage 60.93% (270921/444628)
Region Coverage 55.71% (225504/404793)
Branch Coverage 57.54% (97065/168700)

hello-stephen avatar Dec 19 '25 09:12 hello-stephen