Clean up stats propagation for slicing

[gatesn]

Reduces the amount we copy some stats (by removing into_iter that forces a full stats copy), and also makes duckdb databases idempotent.

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #3356 will improve performances by 31.13%

_{Comparing ngates/slice-stats (ad5a24b) with develop (e5ae65d)}

Summary

⚡ 2 improvements
✅ 817 untouched benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
⚡	canonical_into_nullable[(10000, 10, 0.0)]	344.5 µs	262.7 µs	+31.13%
⚡	into_canonical_non_nullable[(10000, 1, 0.0)]	36.2 µs	28.6 µs	+26.81%

[coveralls]

coverage: 82.243% (-0.008%) from 82.251% when pulling ad5a24b0ba2bf3a5e887c10eb52bce3856e78847 on ngates/slice-stats into e5ae65dd2c6b47fe1b16a38cedde29dc7932f95b on develop.

[gatesn]

I have no recollection of this PR 😊

[github-actions[bot]]

Benchmarks: random_access

Table of Results

name	PR 41015f2b	base c6783bb1	ratio (PR/base)	unit
random-access/vortex-tokio-local-disk	2196417	2.14886e+06	1.02213	ns
random-access/parquet-tokio-local-disk	186050907	1.90806e+08	0.975077	ns

[github-actions[bot]]

Benchmarks: TPC-H on NVME

Table of Results

name	PR 41015f2b	base c6783bb1	ratio (PR/base)	unit
tpch_q01/DataFusion:arrow	62443716	6.224e+07	1.00327	ns
tpch_q02/DataFusion:arrow	16743337	1.73085e+07	0.967345	ns
tpch_q03/DataFusion:arrow	21667799	2.22078e+07	0.975685	ns
tpch_q04/DataFusion:arrow	12046193	1.20174e+07	1.0024	ns
tpch_q05/DataFusion:arrow	31947104	3.14812e+07	1.0148	ns
tpch_q06/DataFusion:arrow	7393802	7.23803e+06	1.02152	ns
tpch_q07/DataFusion:arrow	55724492	5.69791e+07	0.977981	ns
tpch_q08/DataFusion:arrow	18397610	1.82062e+07	1.01051	ns
tpch_q09/DataFusion:arrow	33082001	3.38515e+07	0.977267	ns
tpch_q10/DataFusion:arrow	29388432	2.9271e+07	1.00401	ns
tpch_q11/DataFusion:arrow	7960796	7.72645e+06	1.03033	ns
tpch_q12/DataFusion:arrow	23933546	2.44548e+07	0.978687	ns
tpch_q13/DataFusion:arrow	15147005	1.53713e+07	0.98541	ns
tpch_q14/DataFusion:arrow	6960039	7.1212e+06	0.977369	ns
tpch_q15/DataFusion:arrow	14038619	1.41773e+07	0.990218	ns
tpch_q16/DataFusion:arrow	15212240	1.51077e+07	1.00692	ns
tpch_q17/DataFusion:arrow	57650891	4.88926e+07	1.17913	ns
tpch_q18/DataFusion:arrow	111353362	1.03718e+08	1.07361	ns
tpch_q19/DataFusion:arrow	17089048	1.88619e+07	0.906011	ns
tpch_q20/DataFusion:arrow	20500615	2.19382e+07	0.934471	ns
tpch_q21/DataFusion:arrow	82283938	8.53979e+07	0.963536	ns
tpch_q22/DataFusion:arrow	10119852	1.07164e+07	0.944334	ns
tpch_q01/DataFusion:parquet	158321885	1.75439e+08	0.902433	ns
tpch_q02/DataFusion:parquet	85362277	8.54072e+07	0.999474	ns
tpch_q03/DataFusion:parquet	81365236	8.33192e+07	0.976548	ns
tpch_q04/DataFusion:parquet	56330185	5.83059e+07	0.966115	ns
tpch_q05/DataFusion:parquet	97896151	1.00518e+08	0.973917	ns
tpch_q06/DataFusion:parquet	39179878	3.55446e+07	1.10227	ns
tpch_q07/DataFusion:parquet	117171496	1.18627e+08	0.987733	ns
tpch_q08/DataFusion:parquet	103822601	1.05029e+08	0.988511	ns
tpch_q09/DataFusion:parquet	132107346	1.34666e+08	0.981002	ns
tpch_q10/DataFusion:parquet	118066780	1.28331e+08	0.920015	ns
tpch_q11/DataFusion:parquet	68620439	6.6613e+07	1.03014	ns
tpch_q12/DataFusion:parquet	103303178	8.57042e+07	1.20535	ns
tpch_q13/DataFusion:parquet	165907010	1.73686e+08	0.955215	ns
tpch_q14/DataFusion:parquet	44671521	4.3573e+07	1.02521	ns
tpch_q15/DataFusion:parquet	68935367	6.73432e+07	1.02364	ns
tpch_q16/DataFusion:parquet	47065113	4.86338e+07	0.967744	ns
tpch_q17/DataFusion:parquet	133342882	1.35755e+08	0.982234	ns
tpch_q18/DataFusion:parquet	184338683	1.84819e+08	0.997399	ns
tpch_q19/DataFusion:parquet	82601693	8.51293e+07	0.970308	ns
tpch_q20/DataFusion:parquet	69491856	7.47748e+07	0.929348	ns
tpch_q21/DataFusion:parquet	149421263	1.53585e+08	0.972891	ns
tpch_q22/DataFusion:parquet	76471342	7.69551e+07	0.993713	ns
tpch_q01/DataFusion:vortex-file-compressed	55974770	5.74992e+07	0.973488	ns
tpch_q02/DataFusion:vortex-file-compressed	31365097	3.3665e+07	0.931683	ns
tpch_q03/DataFusion:vortex-file-compressed	21581513	2.23236e+07	0.966756	ns
tpch_q04/DataFusion:vortex-file-compressed	15820853	1.6326e+07	0.969056	ns
tpch_q05/DataFusion:vortex-file-compressed	46672213	4.92465e+07	0.947726	ns
tpch_q06/DataFusion:vortex-file-compressed	7792336	8.37241e+06	0.930716	ns
tpch_q07/DataFusion:vortex-file-compressed	71268800	7.3312e+07	0.97213	ns
tpch_q08/DataFusion:vortex-file-compressed	36740047	3.94184e+07	0.932053	ns
tpch_q09/DataFusion:vortex-file-compressed	52518778	5.16564e+07	1.01669	ns
tpch_q10/DataFusion:vortex-file-compressed	41034851	4.24627e+07	0.966374	ns
tpch_q11/DataFusion:vortex-file-compressed	19233465	1.95576e+07	0.983427	ns
tpch_q12/DataFusion:vortex-file-compressed	16732540	1.75914e+07	0.951177	ns
tpch_q13/DataFusion:vortex-file-compressed	22354268	2.16801e+07	1.0311	ns
tpch_q14/DataFusion:vortex-file-compressed	10379432	1.11129e+07	0.933998	ns
tpch_q15/DataFusion:vortex-file-compressed	18378623	1.91982e+07	0.95731	ns
tpch_q16/DataFusion:vortex-file-compressed	23309159	2.52233e+07	0.924114	ns
tpch_q17/DataFusion:vortex-file-compressed	60434642	5.94125e+07	1.0172	ns
tpch_q18/DataFusion:vortex-file-compressed	95023212	9.48604e+07	1.00172	ns
tpch_q19/DataFusion:vortex-file-compressed	15083997	1.64122e+07	0.919071	ns
tpch_q20/DataFusion:vortex-file-compressed	23237988	2.37029e+07	0.980386	ns
tpch_q21/DataFusion:vortex-file-compressed	71784207	7.42078e+07	0.96734	ns
tpch_q22/DataFusion:vortex-file-compressed	15613323	1.64994e+07	0.946295	ns
tpch_q01/DuckDB:parquet	100771373	1.01457e+08	0.993245	ns
tpch_q02/DuckDB:parquet	45262554	4.32827e+07	1.04574	ns
tpch_q03/DuckDB:parquet	69445223	7.09555e+07	0.978715	ns
tpch_q04/DuckDB:parquet	57636014	5.68288e+07	1.0142	ns
tpch_q05/DuckDB:parquet	69485091	6.89754e+07	1.00739	ns
tpch_q06/DuckDB:parquet	20090870	2.00254e+07	1.00327	ns
tpch_q07/DuckDB:parquet	76381791	7.75979e+07	0.984328	ns
tpch_q08/DuckDB:parquet	88879933	9.07208e+07	0.979708	ns
tpch_q09/DuckDB:parquet	142194202	1.39149e+08	1.02188	ns
tpch_q10/DuckDB:parquet	125437108	1.31838e+08	0.951447	ns
tpch_q11/DuckDB:parquet	23941990	2.31409e+07	1.03462	ns
tpch_q12/DuckDB:parquet	57483802	5.69638e+07	1.00913	ns
tpch_q13/DuckDB:parquet	242851923	2.44869e+08	0.991764	ns
tpch_q14/DuckDB:parquet	44475989	4.67466e+07	0.951428	ns
tpch_q15/DuckDB:parquet	26390853	2.68101e+07	0.984363	ns
tpch_q16/DuckDB:parquet	49288136	4.8302e+07	1.02042	ns
tpch_q17/DuckDB:parquet	59570489	5.41955e+07	1.09918	ns
tpch_q18/DuckDB:parquet	122055713	1.1772e+08	1.03683	ns
tpch_q19/DuckDB:parquet	69923035	7.23087e+07	0.967008	ns
tpch_q20/DuckDB:parquet	60372793	6.09996e+07	0.989724	ns
tpch_q21/DuckDB:parquet	182854835	1.79029e+08	1.02137	ns
tpch_q22/DuckDB:parquet	49831707	4.98818e+07	0.998996	ns
tpch_q01/DuckDB:vortex-file-compressed	31830121	3.32387e+07	0.957622	ns
tpch_q02/DuckDB:vortex-file-compressed	24136574	2.47874e+07	0.973743	ns
tpch_q03/DuckDB:vortex-file-compressed	28800158	2.89799e+07	0.993798	ns
tpch_q04/DuckDB:vortex-file-compressed	27512395	2.8388e+07	0.969154	ns
tpch_q05/DuckDB:vortex-file-compressed	37425600	3.86879e+07	0.967373	ns
tpch_q06/DuckDB:vortex-file-compressed	7765928	7.96755e+06	0.974694	ns
tpch_q07/DuckDB:vortex-file-compressed	34274939	3.36918e+07	1.01731	ns
tpch_q08/DuckDB:vortex-file-compressed	42970326	4.54279e+07	0.945902	ns
tpch_q09/DuckDB:vortex-file-compressed	62625357	6.0239e+07	1.03961	ns
tpch_q10/DuckDB:vortex-file-compressed	39484328	4.01622e+07	0.983121	ns
tpch_q11/DuckDB:vortex-file-compressed	13308639	1.37028e+07	0.971234	ns
tpch_q12/DuckDB:vortex-file-compressed	27824307	2.82319e+07	0.985562	ns
tpch_q13/DuckDB:vortex-file-compressed	39933282	3.90078e+07	1.02373	ns
tpch_q14/DuckDB:vortex-file-compressed	18819154	1.91912e+07	0.980614	ns
tpch_q15/DuckDB:vortex-file-compressed	11914545	1.22255e+07	0.974569	ns
tpch_q16/DuckDB:vortex-file-compressed	23621867	2.4117e+07	0.979469	ns
tpch_q17/DuckDB:vortex-file-compressed	60896271	5.91031e+07	1.03034	ns
tpch_q18/DuckDB:vortex-file-compressed	48324703	4.88546e+07	0.989154	ns
tpch_q19/DuckDB:vortex-file-compressed	23175473	2.24742e+07	1.0312	ns
tpch_q20/DuckDB:vortex-file-compressed	29062743	2.93966e+07	0.988643	ns
tpch_q21/DuckDB:vortex-file-compressed	101875149	1.07923e+08	0.943961	ns
tpch_q22/DuckDB:vortex-file-compressed	20218082	2.10975e+07	0.958318	ns
tpch_q01/DuckDB:duckdb	14314196	1.42287e+07	1.00601	ns
tpch_q02/DuckDB:duckdb	11378129	1.03397e+07	1.10043	ns
tpch_q03/DuckDB:duckdb	15038844	1.4885e+07	1.01034	ns
tpch_q04/DuckDB:duckdb	16458155	1.64632e+07	0.999693	ns
tpch_q05/DuckDB:duckdb	16445620	1.6468e+07	0.998644	ns
tpch_q06/DuckDB:duckdb	3407387	3.42446e+06	0.995016	ns
tpch_q07/DuckDB:duckdb	19168523	1.89949e+07	1.00914	ns
tpch_q08/DuckDB:duckdb	18242455	1.80535e+07	1.01047	ns
tpch_q09/DuckDB:duckdb	54412112	5.36969e+07	1.01332	ns
tpch_q10/DuckDB:duckdb	40514186	4.42333e+07	0.91592	ns
tpch_q11/DuckDB:duckdb	7345698	7.62577e+06	0.963273	ns
tpch_q12/DuckDB:duckdb	12757540	1.29865e+07	0.982366	ns
tpch_q13/DuckDB:duckdb	47667265	4.25183e+07	1.1211	ns
tpch_q14/DuckDB:duckdb	12632521	1.28643e+07	0.981982	ns
tpch_q15/DuckDB:duckdb	7953158	7.72989e+06	1.02888	ns
tpch_q16/DuckDB:duckdb	20105198	1.88476e+07	1.06672	ns
tpch_q17/DuckDB:duckdb	14325716	1.46163e+07	0.980122	ns
tpch_q18/DuckDB:duckdb	35194587	3.57214e+07	0.985251	ns
tpch_q19/DuckDB:duckdb	20502129	2.07756e+07	0.986835	ns
tpch_q20/DuckDB:duckdb	16471246	1.63771e+07	1.00575	ns
tpch_q21/DuckDB:duckdb	50956593	5.06006e+07	1.00704	ns
tpch_q22/DuckDB:duckdb	24024888	2.38766e+07	1.00621	ns

[github-actions[bot]]

Benchmarks: TPC-H on S3

Table of Results

name	PR 41015f2b	base c6783bb1	ratio (PR/base)	unit
tpch_q01/DataFusion:parquet	380323881	3.80618e+08	0.999228	ns
tpch_q02/DataFusion:parquet	790776158	7.62797e+08	1.03668	ns
tpch_q03/DataFusion:parquet	549068446	5.51727e+08	0.995181	ns
tpch_q04/DataFusion:parquet	285051985	2.93109e+08	0.97251	ns
tpch_q05/DataFusion:parquet	692048953	7.13305e+08	0.9702	ns
tpch_q06/DataFusion:parquet	266649836	2.5941e+08	1.02791	ns
tpch_q07/DataFusion:parquet	713128080	7.15541e+08	0.996628	ns
tpch_q08/DataFusion:parquet	813346385	7.7716e+08	1.04656	ns
tpch_q09/DataFusion:parquet	696265110	6.79827e+08	1.02418	ns
tpch_q10/DataFusion:parquet	738910099	7.39496e+08	0.999208	ns
tpch_q11/DataFusion:parquet	535629912	5.3435e+08	1.0024	ns
tpch_q12/DataFusion:parquet	419527116	3.85658e+08	1.08782	ns
tpch_q13/DataFusion:parquet	479361401	4.74665e+08	1.00989	ns
tpch_q14/DataFusion:parquet	343952316	3.25068e+08	1.05809	ns
tpch_q15/DataFusion:parquet	562881531	5.71167e+08	0.985494	ns
tpch_q16/DataFusion:parquet	333703592	3.52575e+08	0.946476	ns
tpch_q17/DataFusion:parquet	551239036	5.33632e+08	1.033	ns
tpch_q18/DataFusion:parquet	695042785	6.97543e+08	0.996416	ns
tpch_q19/DataFusion:parquet	431071306	4.36181e+08	0.988285	ns
tpch_q20/DataFusion:parquet	542893708	5.46266e+08	0.993826	ns
tpch_q21/DataFusion:parquet	839375648	8.50272e+08	0.987184	ns
tpch_q22/DataFusion:parquet	358146291	3.46004e+08	1.03509	ns
tpch_q01/DataFusion:vortex-file-compressed	158348978	1.48214e+08	1.06838	ns
tpch_q02/DataFusion:vortex-file-compressed	143276237	1.29002e+08	1.11065	ns
tpch_q03/DataFusion:vortex-file-compressed	252270982	2.05246e+08	1.22912	ns
tpch_q04/DataFusion:vortex-file-compressed	166305009	1.36571e+08	1.21772	ns
tpch_q05/DataFusion:vortex-file-compressed	241319045	2.06475e+08	1.16876	ns
tpch_q06/DataFusion:vortex-file-compressed	135990532	1.17141e+08	1.16091	ns
tpch_q07/DataFusion:vortex-file-compressed	251538099	2.57678e+08	0.976174	ns
tpch_q08/DataFusion:vortex-file-compressed	283905003	2.55156e+08	1.11267	ns
tpch_q09/DataFusion:vortex-file-compressed	267282493	2.60616e+08	1.02558	ns
tpch_q10/DataFusion:vortex-file-compressed	284070525	2.4862e+08	1.14259	ns
tpch_q11/DataFusion:vortex-file-compressed	130058314	1.23368e+08	1.05423	ns
tpch_q12/DataFusion:vortex-file-compressed	193967605	1.78771e+08	1.085	ns
tpch_q13/DataFusion:vortex-file-compressed	120186970	1.27101e+08	0.9456	ns
tpch_q14/DataFusion:vortex-file-compressed	140951285	1.31284e+08	1.07363	ns
tpch_q15/DataFusion:vortex-file-compressed	217261470	2.10577e+08	1.03174	ns
tpch_q16/DataFusion:vortex-file-compressed	99127589	7.69013e+07	1.28902	ns
tpch_q17/DataFusion:vortex-file-compressed	242001147	2.24122e+08	1.07977	ns
tpch_q18/DataFusion:vortex-file-compressed	293162028	2.83605e+08	1.0337	ns
tpch_q19/DataFusion:vortex-file-compressed	158368131	1.65389e+08	0.95755	ns
tpch_q20/DataFusion:vortex-file-compressed	215918369	2.18646e+08	0.987527	ns
tpch_q21/DataFusion:vortex-file-compressed	382170744	3.6898e+08	1.03575	ns
tpch_q22/DataFusion:vortex-file-compressed	96185130	1.18558e+08	0.811292	ns
tpch_q01/DuckDB:parquet	115090834	1.39879e+08	0.822789	ns
tpch_q02/DuckDB:parquet	339702345	3.18946e+08	1.06508	ns
tpch_q03/DuckDB:parquet	157863753	1.65893e+08	0.951599	ns
tpch_q04/DuckDB:parquet	104210615	1.25864e+08	0.827962	ns
tpch_q05/DuckDB:parquet	205276494	2.66417e+08	0.770508	ns
tpch_q06/DuckDB:parquet	38719353	5.7292e+07	0.675825	ns
tpch_q07/DuckDB:parquet	272146797	2.91431e+08	0.933828	ns
tpch_q08/DuckDB:parquet	381886768	4.00482e+08	0.953568	ns
tpch_q09/DuckDB:parquet	342427470	3.13783e+08	1.09129	ns
tpch_q10/DuckDB:parquet	279129257	2.57754e+08	1.08293	ns
tpch_q11/DuckDB:parquet	190160774	2.46651e+08	0.77097	ns
tpch_q12/DuckDB:parquet	114550936	1.27046e+08	0.901652	ns
tpch_q13/DuckDB:parquet	297196522	3.18744e+08	0.932399	ns
tpch_q14/DuckDB:parquet	116333573	1.02735e+08	1.13237	ns
tpch_q15/DuckDB:parquet	85427510	1.01039e+08	0.845489	ns
tpch_q16/DuckDB:parquet	143745223	1.46275e+08	0.982702	ns
tpch_q17/DuckDB:parquet	163700451	1.75375e+08	0.933432	ns
tpch_q18/DuckDB:parquet	251883208	2.80124e+08	0.899186	ns
tpch_q19/DuckDB:parquet	97897440	1.21777e+08	0.803907	ns
tpch_q20/DuckDB:parquet	211209611	2.01759e+08	1.04684	ns
tpch_q21/DuckDB:parquet	321684838	3.47223e+08	0.92645	ns
tpch_q22/DuckDB:parquet	132045073	1.5686e+08	0.841803	ns

[github-actions[bot]]

Benchmarks: Clickbench on NVME

Table of Results

name	PR 41015f2b	base c6783bb1	ratio (PR/base)	unit
clickbench_q00/DataFusion:parquet	1629076	1.66781e+06	0.976776	ns
clickbench_q01/DataFusion:parquet	27512370	2.84854e+07	0.96584	ns
clickbench_q02/DataFusion:parquet	64130647	6.26539e+07	1.02357	ns
clickbench_q03/DataFusion:parquet	57737505	5.71596e+07	1.01011	ns
clickbench_q04/DataFusion:parquet	363582801	3.7039e+08	0.981622	ns
clickbench_q05/DataFusion:parquet	388013498	3.85031e+08	1.00775	ns
clickbench_q06/DataFusion:parquet	1620052	1.63535e+06	0.990644	ns
clickbench_q07/DataFusion:parquet	31217585	3.24952e+07	0.960682	ns
clickbench_q08/DataFusion:parquet	455592895	4.66605e+08	0.9764	ns
clickbench_q09/DataFusion:parquet	675949692	6.92128e+08	0.976626	ns
clickbench_q10/DataFusion:parquet	148180140	1.50734e+08	0.983055	ns
clickbench_q11/DataFusion:parquet	174269778	1.70736e+08	1.0207	ns
clickbench_q12/DataFusion:parquet	412487330	4.15545e+08	0.992642	ns
clickbench_q13/DataFusion:parquet	588520838	5.93015e+08	0.992422	ns
clickbench_q14/DataFusion:parquet	397498243	3.94938e+08	1.00648	ns
clickbench_q15/DataFusion:parquet	445893728	4.49225e+08	0.992585	ns
clickbench_q16/DataFusion:parquet	835994954	8.2768e+08	1.01005	ns
clickbench_q17/DataFusion:parquet	793411377	7.94153e+08	0.999066	ns
clickbench_q18/DataFusion:parquet	1565978074	1.58699e+09	0.986757	ns
clickbench_q19/DataFusion:parquet	45393663	4.70611e+07	0.964569	ns
clickbench_q20/DataFusion:parquet	653114576	6.5657e+08	0.994738	ns
clickbench_q21/DataFusion:parquet	754240107	7.52859e+08	1.00183	ns
clickbench_q22/DataFusion:parquet	1085950701	1.10236e+09	0.985119	ns
clickbench_q23/DataFusion:parquet	4831296978	4.83488e+09	0.999259	ns
clickbench_q24/DataFusion:parquet	252096665	2.57388e+08	0.979441	ns
clickbench_q25/DataFusion:parquet	218541614	2.16654e+08	1.00871	ns
clickbench_q26/DataFusion:parquet	288831741	2.87071e+08	1.00613	ns
clickbench_q27/DataFusion:parquet	1059376196	1.08663e+09	0.974915	ns
clickbench_q28/DataFusion:parquet	5807060573	5.87918e+09	0.987733	ns
clickbench_q29/DataFusion:parquet	250575169	2.44968e+08	1.02289	ns
clickbench_q30/DataFusion:parquet	383699646	3.95617e+08	0.969875	ns
clickbench_q31/DataFusion:parquet	442584270	4.41801e+08	1.00177	ns
clickbench_q32/DataFusion:parquet	1360743380	1.35672e+09	1.00297	ns
clickbench_q33/DataFusion:parquet	1690454283	1.70132e+09	0.993611	ns
clickbench_q34/DataFusion:parquet	1695365000	1.70098e+09	0.9967	ns
clickbench_q35/DataFusion:parquet	621622276	6.23649e+08	0.996751	ns
clickbench_q36/DataFusion:parquet	180264244	1.80178e+08	1.00048	ns
clickbench_q37/DataFusion:parquet	70444202	7.27641e+07	0.968118	ns
clickbench_q38/DataFusion:parquet	99209943	1.06524e+08	0.931339	ns
clickbench_q39/DataFusion:parquet	336045698	3.3484e+08	1.0036	ns
clickbench_q40/DataFusion:parquet	44938138	3.95015e+07	1.13763	ns
clickbench_q41/DataFusion:parquet	39518774	3.74232e+07	1.056	ns
clickbench_q42/DataFusion:parquet	54228751	5.34194e+07	1.01515	ns
clickbench_q00/DataFusion:vortex-file-compressed	1664141	1.72415e+06	0.965195	ns
clickbench_q01/DataFusion:vortex-file-compressed	9547867	8.74879e+06	1.09134	ns
clickbench_q02/DataFusion:vortex-file-compressed	27656706	2.8213e+07	0.980282	ns
clickbench_q03/DataFusion:vortex-file-compressed	33704902	3.60633e+07	0.934604	ns
clickbench_q04/DataFusion:vortex-file-compressed	323528309	3.18507e+08	1.01577	ns
clickbench_q05/DataFusion:vortex-file-compressed	338419360	3.48696e+08	0.970529	ns
clickbench_q06/DataFusion:vortex-file-compressed	1719428	1.71781e+06	1.00094	ns
clickbench_q07/DataFusion:vortex-file-compressed	14094423	1.28069e+07	1.10053	ns
clickbench_q08/DataFusion:vortex-file-compressed	393494328	3.87464e+08	1.01556	ns
clickbench_q09/DataFusion:vortex-file-compressed	569284995	5.96559e+08	0.954282	ns
clickbench_q10/DataFusion:vortex-file-compressed	86391257	8.9787e+07	0.96218	ns
clickbench_q11/DataFusion:vortex-file-compressed	98476199	9.93407e+07	0.991298	ns
clickbench_q12/DataFusion:vortex-file-compressed	292262049	3.17777e+08	0.919708	ns
clickbench_q13/DataFusion:vortex-file-compressed	494312717	5.02793e+08	0.983135	ns
clickbench_q14/DataFusion:vortex-file-compressed	284726814	2.84442e+08	1.001	ns
clickbench_q15/DataFusion:vortex-file-compressed	390497833	3.84447e+08	1.01574	ns
clickbench_q16/DataFusion:vortex-file-compressed	836010970	8.21971e+08	1.01708	ns
clickbench_q17/DataFusion:vortex-file-compressed	793546444	7.92219e+08	1.00168	ns
clickbench_q18/DataFusion:vortex-file-compressed	1491155337	1.51701e+09	0.982957	ns
clickbench_q19/DataFusion:vortex-file-compressed	18074759	1.74259e+07	1.03724	ns
clickbench_q20/DataFusion:vortex-file-compressed	309818973	3.57178e+08	0.867409	ns
clickbench_q21/DataFusion:vortex-file-compressed	389271886	4.16652e+08	0.934285	ns
clickbench_q22/DataFusion:vortex-file-compressed	723092053	8.35714e+08	0.865239	ns
clickbench_q23/DataFusion:vortex-file-compressed	2697816565	3.07022e+09	0.878705	ns
clickbench_q24/DataFusion:vortex-file-compressed	143484189	1.45512e+08	0.986061	ns
clickbench_q25/DataFusion:vortex-file-compressed	112242265	1.18587e+08	0.946496	ns
clickbench_q26/DataFusion:vortex-file-compressed	175050248	1.81391e+08	0.965042	ns
clickbench_q27/DataFusion:vortex-file-compressed	808394796	8.35934e+08	0.967055	ns
clickbench_q28/DataFusion:vortex-file-compressed	6775626066	6.38752e+09	1.06076	ns
clickbench_q29/DataFusion:vortex-file-compressed	246742657	2.34717e+08	1.05124	ns
clickbench_q30/DataFusion:vortex-file-compressed	267983014	2.74579e+08	0.975978	ns
clickbench_q31/DataFusion:vortex-file-compressed	308843049	3.17134e+08	0.973857	ns
clickbench_q32/DataFusion:vortex-file-compressed	1464397091	1.40286e+09	1.04386	ns
clickbench_q33/DataFusion:vortex-file-compressed	1572613576	1.60631e+09	0.979025	ns
clickbench_q34/DataFusion:vortex-file-compressed	1566891740	1.68414e+09	0.930382	ns
clickbench_q35/DataFusion:vortex-file-compressed	571615393	5.64997e+08	1.01171	ns
clickbench_q36/DataFusion:vortex-file-compressed	106520776	1.00586e+08	1.059	ns
clickbench_q37/DataFusion:vortex-file-compressed	55993068	4.89202e+07	1.14458	ns
clickbench_q38/DataFusion:vortex-file-compressed	43541805	3.69471e+07	1.17849	ns
clickbench_q39/DataFusion:vortex-file-compressed	220841058	2.1704e+08	1.01751	ns
clickbench_q40/DataFusion:vortex-file-compressed	25428147	2.07844e+07	1.22342	ns
clickbench_q41/DataFusion:vortex-file-compressed	23590179	2.03343e+07	1.16012	ns
clickbench_q42/DataFusion:vortex-file-compressed	37764379	3.42614e+07	1.10224	ns
clickbench_q00/DuckDB:parquet	129180762	1.30483e+08	0.990017	ns
clickbench_q01/DuckDB:parquet	53516498	5.43707e+07	0.98429	ns
clickbench_q02/DuckDB:parquet	69804283	6.95233e+07	1.00404	ns
clickbench_q03/DuckDB:parquet	67761599	6.79344e+07	0.997456	ns
clickbench_q04/DuckDB:parquet	216143999	2.15655e+08	1.00227	ns
clickbench_q05/DuckDB:parquet	242574908	2.43912e+08	0.99452	ns
clickbench_q06/DuckDB:parquet	71118972	7.3407e+07	0.968831	ns
clickbench_q07/DuckDB:parquet	63074731	6.54726e+07	0.963375	ns
clickbench_q08/DuckDB:parquet	282711266	2.80476e+08	1.00797	ns
clickbench_q09/DuckDB:parquet	411162920	4.12967e+08	0.995631	ns
clickbench_q10/DuckDB:parquet	109254796	1.11803e+08	0.977206	ns
clickbench_q11/DuckDB:parquet	122704150	1.22775e+08	0.999425	ns
clickbench_q12/DuckDB:parquet	259282638	2.59413e+08	0.999498	ns
clickbench_q13/DuckDB:parquet	419471450	4.22165e+08	0.99362	ns
clickbench_q14/DuckDB:parquet	289160645	2.92705e+08	0.987891	ns
clickbench_q15/DuckDB:parquet	244536527	2.44673e+08	0.999443	ns
clickbench_q16/DuckDB:parquet	579932031	5.85296e+08	0.990835	ns
clickbench_q17/DuckDB:parquet	498339358	4.99598e+08	0.99748	ns
clickbench_q18/DuckDB:parquet	977485905	9.73605e+08	1.00399	ns
clickbench_q19/DuckDB:parquet	63924824	6.63682e+07	0.963185	ns
clickbench_q20/DuckDB:parquet	380876564	3.96303e+08	0.961075	ns
clickbench_q21/DuckDB:parquet	505349794	5.06432e+08	0.997864	ns
clickbench_q22/DuckDB:parquet	878677272	8.75215e+08	1.00396	ns
clickbench_q23/DuckDB:parquet	290826467	2.84673e+08	1.02162	ns
clickbench_q24/DuckDB:parquet	111552117	1.10601e+08	1.0086	ns
clickbench_q25/DuckDB:parquet	143387412	1.42291e+08	1.0077	ns
clickbench_q26/DuckDB:parquet	64733128	6.62802e+07	0.976659	ns
clickbench_q27/DuckDB:parquet	636714399	6.41379e+08	0.992727	ns
clickbench_q28/DuckDB:parquet	5693395368	5.69289e+09	1.00009	ns
clickbench_q29/DuckDB:parquet	77500776	7.52286e+07	1.0302	ns
clickbench_q30/DuckDB:parquet	297091643	2.99533e+08	0.991848	ns
clickbench_q31/DuckDB:parquet	343322299	3.45777e+08	0.9929	ns
clickbench_q32/DuckDB:parquet	1033157618	1.03918e+09	0.994206	ns
clickbench_q33/DuckDB:parquet	1064976410	1.05769e+09	1.00689	ns
clickbench_q34/DuckDB:parquet	1135953295	1.13674e+09	0.999308	ns
clickbench_q35/DuckDB:parquet	371590706	3.83607e+08	0.968675	ns
clickbench_q36/DuckDB:parquet	68365267	6.94249e+07	0.984737	ns
clickbench_q37/DuckDB:parquet	58844081	5.83435e+07	1.00858	ns
clickbench_q38/DuckDB:parquet	69647970	6.91593e+07	1.00707	ns
clickbench_q39/DuckDB:parquet	99237862	1.03681e+08	0.957148	ns
clickbench_q40/DuckDB:parquet	64844175	6.59045e+07	0.983911	ns
clickbench_q41/DuckDB:parquet	77263266	7.68656e+07	1.00517	ns
clickbench_q42/DuckDB:parquet	57097590	5.80129e+07	0.984222	ns
clickbench_q00/DuckDB:vortex-file-compressed	10639247	9.77047e+06	1.08892	ns
clickbench_q01/DuckDB:vortex-file-compressed	13941931	1.32664e+07	1.05092	ns
clickbench_q02/DuckDB:vortex-file-compressed	28496550	2.83197e+07	1.00624	ns
clickbench_q03/DuckDB:vortex-file-compressed	34160577	3.35874e+07	1.01706	ns
clickbench_q04/DuckDB:vortex-file-compressed	209912214	2.14316e+08	0.979453	ns
clickbench_q05/DuckDB:vortex-file-compressed	163845241	1.63264e+08	1.00356	ns
clickbench_q06/DuckDB:vortex-file-compressed	37943242	3.87425e+07	0.979371	ns
clickbench_q07/DuckDB:vortex-file-compressed	18949608	1.81193e+07	1.04582	ns
clickbench_q08/DuckDB:vortex-file-compressed	278128034	2.74373e+08	1.01369	ns
clickbench_q09/DuckDB:vortex-file-compressed	396663907	3.98455e+08	0.995504	ns
clickbench_q10/DuckDB:vortex-file-compressed	90933617	8.92994e+07	1.0183	ns
clickbench_q11/DuckDB:vortex-file-compressed	98381213	1.00873e+08	0.975297	ns
clickbench_q12/DuckDB:vortex-file-compressed	182087951	1.82233e+08	0.999207	ns
clickbench_q13/DuckDB:vortex-file-compressed	361457290	3.65261e+08	0.989587	ns
clickbench_q14/DuckDB:vortex-file-compressed	208070236	2.08191e+08	0.999418	ns
clickbench_q15/DuckDB:vortex-file-compressed	241259397	2.41403e+08	0.999403	ns
clickbench_q16/DuckDB:vortex-file-compressed	514740670	5.18468e+08	0.992811	ns
clickbench_q17/DuckDB:vortex-file-compressed	439309330	4.35343e+08	1.00911	ns
clickbench_q18/DuckDB:vortex-file-compressed	892176398	9.06503e+08	0.984195	ns
clickbench_q19/DuckDB:vortex-file-compressed	20545073	2.08799e+07	0.983962	ns
clickbench_q20/DuckDB:vortex-file-compressed	294358522	2.92278e+08	1.00712	ns
clickbench_q21/DuckDB:vortex-file-compressed	368293526	3.57073e+08	1.03142	ns
clickbench_q22/DuckDB:vortex-file-compressed	564432637	5.84012e+08	0.966474	ns
clickbench_q23/DuckDB:vortex-file-compressed	290320840	2.93948e+08	0.987661	ns
clickbench_q24/DuckDB:vortex-file-compressed	24660379	2.52931e+07	0.974985	ns
clickbench_q25/DuckDB:vortex-file-compressed	82461600	8.42505e+07	0.978767	ns
clickbench_q26/DuckDB:vortex-file-compressed	22628444	2.39565e+07	0.944562	ns
clickbench_q27/DuckDB:vortex-file-compressed	544810871	5.51976e+08	0.987019	ns
clickbench_q28/DuckDB:vortex-file-compressed	5553984723	5.5668e+09	0.997698	ns
clickbench_q29/DuckDB:vortex-file-compressed	32558590	3.17616e+07	1.02509	ns
clickbench_q30/DuckDB:vortex-file-compressed	186071601	1.90855e+08	0.974937	ns
clickbench_q31/DuckDB:vortex-file-compressed	248254126	2.57009e+08	0.965935	ns
clickbench_q32/DuckDB:vortex-file-compressed	1030963988	1.04826e+09	0.983502	ns
clickbench_q33/DuckDB:vortex-file-compressed	933964705	9.3305e+08	1.00098	ns
clickbench_q34/DuckDB:vortex-file-compressed	1044618236	1.05757e+09	0.98775	ns
clickbench_q35/DuckDB:vortex-file-compressed	391573805	3.895e+08	1.00532	ns
clickbench_q36/DuckDB:vortex-file-compressed	23521939	2.01668e+07	1.16637	ns
clickbench_q37/DuckDB:vortex-file-compressed	15630654	1.44035e+07	1.0852	ns
clickbench_q38/DuckDB:vortex-file-compressed	16261996	1.53122e+07	1.06203	ns
clickbench_q39/DuckDB:vortex-file-compressed	39724090	3.53e+07	1.12533	ns
clickbench_q40/DuckDB:vortex-file-compressed	14007873	1.21057e+07	1.15713	ns
clickbench_q41/DuckDB:vortex-file-compressed	14691689	1.41143e+07	1.04091	ns
clickbench_q42/DuckDB:vortex-file-compressed	20471942	1.71128e+07	1.19629	ns
clickbench_q00/DuckDB:duckdb	13877170	1.79154e+07	0.774596	ns
clickbench_q01/DuckDB:duckdb	5289156	5.58488e+06	0.94705	ns
clickbench_q02/DuckDB:duckdb	14304996	1.52085e+07	0.940593	ns
clickbench_q03/DuckDB:duckdb	19448539	1.97331e+07	0.985581	ns
clickbench_q04/DuckDB:duckdb	199679953	2.20822e+08	0.904256	ns
clickbench_q05/DuckDB:duckdb	241672385	2.44645e+08	0.987848	ns
clickbench_q06/DuckDB:duckdb	14541497	1.58699e+07	0.916294	ns
clickbench_q07/DuckDB:duckdb	5503851	5.73868e+06	0.95908	ns
clickbench_q08/DuckDB:duckdb	252955776	2.54669e+08	0.993272	ns
clickbench_q09/DuckDB:duckdb	353955965	3.5586e+08	0.99465	ns
clickbench_q10/DuckDB:duckdb	78041357	7.93807e+07	0.983128	ns
clickbench_q11/DuckDB:duckdb	84572284	8.55864e+07	0.988151	ns
clickbench_q12/DuckDB:duckdb	204309359	2.06888e+08	0.987534	ns
clickbench_q13/DuckDB:duckdb	348003366	3.53966e+08	0.983155	ns
clickbench_q14/DuckDB:duckdb	216796863	2.20652e+08	0.982527	ns
clickbench_q15/DuckDB:duckdb	223431232	2.26282e+08	0.987401	ns
clickbench_q16/DuckDB:duckdb	511267859	5.14391e+08	0.993929	ns
clickbench_q17/DuckDB:duckdb	424852674	4.33963e+08	0.979007	ns
clickbench_q18/DuckDB:duckdb	845769147	8.47346e+08	0.99814	ns
clickbench_q19/DuckDB:duckdb	2806113	2.44108e+06	1.14954	ns
clickbench_q20/DuckDB:duckdb	287081733	3.05138e+08	0.940825	ns
clickbench_q21/DuckDB:duckdb	252124424	2.62196e+08	0.961586	ns
clickbench_q22/DuckDB:duckdb	387231017	3.95234e+08	0.97975	ns
clickbench_q23/DuckDB:duckdb	46604603	4.68259e+07	0.995274	ns
clickbench_q24/DuckDB:duckdb	15278023	1.64913e+07	0.926431	ns
clickbench_q25/DuckDB:duckdb	78049363	7.85817e+07	0.993226	ns
clickbench_q26/DuckDB:duckdb	13956657	1.3979e+07	0.998404	ns
clickbench_q27/DuckDB:duckdb	379078619	3.89431e+08	0.973417	ns
clickbench_q28/DuckDB:duckdb	5512280029	5.52743e+09	0.99726	ns
clickbench_q29/DuckDB:duckdb	22184540	2.25382e+07	0.984308	ns
clickbench_q30/DuckDB:duckdb	198318673	1.96885e+08	1.00728	ns
clickbench_q31/DuckDB:duckdb	236661464	2.38128e+08	0.993843	ns
clickbench_q32/DuckDB:duckdb	981336036	9.91623e+08	0.989626	ns
clickbench_q33/DuckDB:duckdb	858000121	8.81319e+08	0.973541	ns
clickbench_q34/DuckDB:duckdb	944183392	9.57822e+08	0.985761	ns
clickbench_q35/DuckDB:duckdb	297921580	2.92492e+08	1.01856	ns
clickbench_q36/DuckDB:duckdb	31125853	3.12724e+07	0.995313	ns
clickbench_q37/DuckDB:duckdb	4582005	1.24147e+07	0.369078	ns
clickbench_q38/DuckDB:duckdb	7318905	8.60068e+06	0.850968	ns
clickbench_q39/DuckDB:duckdb	45886555	4.57908e+07	1.00209	ns
clickbench_q40/DuckDB:duckdb	5094493	5.56111e+06	0.916093	ns
clickbench_q41/DuckDB:duckdb	4337047	5.29792e+06	0.818632	ns
clickbench_q42/DuckDB:duckdb	8529401	1.11773e+07	0.763102	ns

[a10y]

@claude is this true

[claude[bot]]

Claude finished @a10y's task —— View job

---

Analysis: Is This PR Description Accurate?

Based on my code review, here's what I can verify about the PR claims:

✅ TRUE: Reduces stats copying by removing into_iter

Evidence found:

• Slicing optimization (vortex-array/src/array/mod.rs:399): Changed from into_iter() to with_iter() when propagating stats to sliced arrays
• New borrowed iterator pattern (stats/array.rs:111-120): The with_iter() method provides a borrowed iterator within a closure, preventing full stats set cloning
• TUI display fix (vortex-tui/src/browse/ui/layouts.rs:165): Avoided copying stats when displaying them in the terminal UI

This optimization prevents unnecessary deep copies of statistics data structures during array slicing operations.

✅ TRUE: Performance improvements

Evidence from CodSpeed benchmarks:

• canonical_into_nullable[(10000, 10, 0.0)]: +31.13% improvement (344.5 µs → 262.7 µs)
• into_canonical_non_nullable[(10000, 1, 0.0)]: +26.81% improvement (36.2 µs → 28.6 µs)

These improvements align with the reduced copying - less time spent cloning stats data structures.

❓ UNCLEAR: DuckDB databases idempotent claim

What I found:

• No DuckDB-related changes in the main PR files
• There's a separate commit (9044182) about optionally deleting DuckDB database files, but it's not part of this PR's core changes
• The "idempotent" claim may refer to something else or be inaccurate/misplaced

Overall Assessment

The main claims about reducing stats copying and improving performance are TRUE and well-supported by the code changes. The DuckDB idempotent claim appears to be either inaccurate or referring to changes not clearly visible in this PR.