doris icon indicating copy to clipboard operation
doris copied to clipboard
verified publisher icon apache

[fix](filecache) fix benchmark varification problem

Open freemandealer opened this issue 2 weeks ago • 12 comments

Cause of the the problem: The generator reused one 1 MB buffer, but the writer batched up to 5 MB before calling appendv, so every pending Slice pointed at the same buffer that kept being overwritten; the uploaded data no longer matched the key/offset tags, making DataVerifier fail.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • [ ] Regression test
    • [ ] Unit Test
    • [ ] Manual test (add detailed scripts or steps below)
    • [ ] No need to test or manual test. Explain why:
      • [ ] This is a refactor/code format and no logic has been changed.
      • [ ] Previous test can cover this change.
      • [ ] No code files have been changed.
      • [ ] Other reason

  • Behavior changed:

    • [ ] No.
    • [ ] Yes.

  • Does this need documentation?

    • [ ] No.
    • [ ] Yes.

Check List (For Reviewer who merge this PR)

  • [ ] Confirm the release note
  • [ ] Confirm test cases
  • [ ] Confirm document
  • [ ] Add branch pick label

freemandealer avatar Dec 09 '25 07:12 freemandealer

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Thearas avatar Dec 09 '25 07:12 Thearas

run buildall

freemandealer avatar Dec 09 '25 07:12 freemandealer

BE UT Coverage Report

Increment line coverage :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 53.35% (18708/35065)
Line Coverage 39.05% (172943/442854)
Region Coverage 33.73% (134134/397726)
Branch Coverage 34.64% (57642/166416)

hello-stephen avatar Dec 09 '25 10:12 hello-stephen

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 72.23% (24825/34368)
Line Coverage 58.94% (260766/442398)
Region Coverage 53.93% (217058/402454)
Branch Coverage 55.35% (92604/167293)

hello-stephen avatar Dec 09 '25 13:12 hello-stephen

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Dec 09 '25 22:12 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Dec 09 '25 22:12 github-actions[bot]

run p0

freemandealer avatar Dec 10 '25 16:12 freemandealer

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 72.23% (24825/34368)
Line Coverage 58.94% (260744/442398)
Region Coverage 53.90% (216924/402454)
Branch Coverage 55.35% (92592/167293)

hello-stephen avatar Dec 10 '25 17:12 hello-stephen

run performance

freemandealer avatar Dec 11 '25 01:12 freemandealer

TPC-H: Total hot run time: 35449 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 976a2b3baa0b62d94ab297bf2fb8f5e175a0b0df, data reload: false

------ Round 1 ----------------------------------
q1	17617	4206	4025	4025
q2	2007	355	235	235
q3	10195	1322	716	716
q4	10227	952	318	318
q5	7561	2143	1909	1909
q6	193	177	147	147
q7	998	855	718	718
q8	9362	1454	1154	1154
q9	6977	5282	5417	5282
q10	6819	2387	1972	1972
q11	537	310	290	290
q12	660	708	564	564
q13	17952	3711	3028	3028
q14	323	315	300	300
q15	601	521	514	514
q16	962	915	877	877
q17	720	838	563	563
q18	7583	7284	7247	7247
q19	1115	971	636	636
q20	415	369	259	259
q21	4246	3960	3715	3715
q22	1051	1010	980	980
Total cold run time: 108121 ms
Total hot run time: 35449 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4065	4069	3995	3995
q2	346	414	313	313
q3	2152	2683	2262	2262
q4	1305	1727	1277	1277
q5	4228	4720	4819	4720
q6	259	186	132	132
q7	2056	1924	1807	1807
q8	2704	2553	2429	2429
q9	7474	7571	7437	7437
q10	3046	3332	2719	2719
q11	628	518	498	498
q12	709	924	684	684
q13	3530	4022	3337	3337
q14	294	307	299	299
q15	562	509	519	509
q16	905	936	873	873
q17	1190	1438	1455	1438
q18	8004	7726	7592	7592
q19	849	822	842	822
q20	1888	1985	1838	1838
q21	4594	4260	4120	4120
q22	1135	1042	983	983
Total cold run time: 51923 ms
Total hot run time: 50084 ms

doris-robot avatar Dec 11 '25 02:12 doris-robot

TPC-DS: Total hot run time: 181713 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 976a2b3baa0b62d94ab297bf2fb8f5e175a0b0df, data reload: false

query5	4834	615	476	476
query6	328	230	215	215
query7	4218	461	284	284
query8	336	258	246	246
query9	8803	2585	2590	2585
query10	529	387	339	339
query11	15306	14769	14663	14663
query12	179	123	116	116
query13	1281	522	410	410
query14	6237	3263	3011	3011
query14_1	2915	2923	2905	2905
query15	216	196	183	183
query16	987	491	494	491
query17	1135	752	611	611
query18	2503	462	363	363
query19	229	234	207	207
query20	123	113	110	110
query21	221	142	119	119
query22	3981	4062	3829	3829
query23	16514	16110	16021	16021
query23_1	16042	16115	16043	16043
query24	7448	1699	1265	1265
query24_1	1252	1255	1268	1255
query25	576	502	486	486
query26	1243	269	164	164
query27	2767	461	304	304
query28	4473	2155	2146	2146
query29	815	547	443	443
query30	318	242	217	217
query31	828	694	604	604
query32	80	64	76	64
query33	554	319	295	295
query34	932	925	543	543
query35	789	811	733	733
query36	853	920	813	813
query37	129	96	75	75
query38	3846	3841	3826	3826
query39	758	727	716	716
query39_1	698	709	708	708
query40	222	135	123	123
query41	64	63	61	61
query42	108	109	109	109
query43	442	442	414	414
query44	1381	759	768	759
query45	195	190	178	178
query46	892	987	623	623
query47	1687	1698	1632	1632
query48	323	351	260	260
query49	632	438	414	414
query50	654	301	220	220
query51	3817	3893	3840	3840
query52	109	112	97	97
query53	321	349	295	295
query54	311	263	270	263
query55	79	75	76	75
query56	288	290	319	290
query57	1166	1135	1074	1074
query58	282	266	249	249
query59	2328	2398	2314	2314
query60	315	312	308	308
query61	164	159	165	159
query62	712	703	633	633
query63	328	292	307	292
query64	5003	1333	1027	1027
query65	4025	3922	3968	3922
query66	1458	443	324	324
query67	15197	14965	14863	14863
query68	5340	1051	742	742
query69	511	345	320	320
query70	1067	1016	980	980
query71	367	321	291	291
query72	5728	4965	5160	4965
query73	654	571	311	311
query74	8777	8828	8609	8609
query75	3558	3506	3150	3150
query76	3946	1140	780	780
query77	518	411	304	304
query78	9438	9644	8858	8858
query79	1843	860	613	613
query80	1392	653	571	571
query81	567	267	236	236
query82	415	131	107	107
query83	287	257	238	238
query84	257	124	102	102
query85	912	514	492	492
query86	416	283	280	280
query87	4097	4101	3981	3981
query88	3294	2290	2279	2279
query89	471	424	394	394
query90	2025	165	153	153
query91	179	164	141	141
query92	79	67	67	67
query93	1273	942	569	569
query94	552	308	274	274
query95	570	332	358	332
query96	609	493	210	210
query97	2595	2659	2559	2559
query98	212	200	189	189
query99	1318	1313	1211	1211
Total cold run time: 261255 ms
Total hot run time: 181713 ms

doris-robot avatar Dec 11 '25 02:12 doris-robot

ClickBench: Total hot run time: 27.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 976a2b3baa0b62d94ab297bf2fb8f5e175a0b0df, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.06	0.05
query3	0.25	0.09	0.08
query4	1.61	0.11	0.11
query5	0.27	0.25	0.27
query6	1.16	0.65	0.63
query7	0.04	0.03	0.02
query8	0.06	0.04	0.04
query9	0.58	0.51	0.49
query10	0.56	0.56	0.56
query11	0.17	0.11	0.10
query12	0.14	0.11	0.11
query13	0.63	0.61	0.60
query14	1.00	0.98	0.98
query15	0.83	0.80	0.79
query16	0.41	0.40	0.40
query17	1.02	1.00	1.01
query18	0.24	0.22	0.22
query19	1.97	1.88	1.89
query20	0.02	0.01	0.01
query21	15.44	0.28	0.14
query22	4.82	0.05	0.05
query23	16.05	0.30	0.10
query24	1.11	0.73	0.55
query25	0.05	0.09	0.06
query26	0.14	0.14	0.13
query27	0.09	0.04	0.06
query28	4.98	1.23	1.02
query29	12.62	3.96	3.24
query30	0.28	0.13	0.14
query31	2.82	0.63	0.40
query32	3.24	0.55	0.45
query33	3.03	3.01	3.05
query34	16.78	5.22	4.51
query35	4.59	4.59	4.58
query36	0.66	0.51	0.49
query37	0.11	0.07	0.07
query38	0.07	0.05	0.03
query39	0.04	0.02	0.03
query40	0.18	0.15	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.38 s
Total hot run time: 27.6 s

doris-robot avatar Dec 11 '25 02:12 doris-robot