vortex
vortex copied to clipboard
spiraldb
feat[layout]: replace register_splits with a layout splits stream
coverage: 81.011% (-0.1%) from 81.109% when pulling b0e97510fece2e3e318ce790caef33e7c4b7fcef on ji/layout-mask-stream into db33b9fe92682497b34e41d6568bac8d831a1e48 on develop.
Benchmarks: random_access
Table of Results
| name | PR 5ddde6b8 | base db33b9fe | ratio (PR/base) | unit |
|---|---|---|---|---|
| random-access/vortex-tokio-local-disk | 2183909 | 2.26503e+06 | 0.964185 | ns |
| random-access/parquet-tokio-local-disk | 194756531 | 1.88713e+08 | 1.03203 | ns |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Benchmarks: compress
Table of Results
| name | PR 5ddde6b8 | base db33b9fe | ratio (PR/base) | unit |
|---|---|---|---|---|
| compress time/taxi | 2.0092e+08 | 1.87863e+08 | 1.0695 | ns |
| parquet_rs-zstd compress time/taxi | 1.65486e+09 | 1.62074e+09 | 1.02105 | ns |
| decompress time/taxi | 1.23461e+08 | 1.23794e+08 | 0.997308 | ns |
| parquet_rs-zstd decompress time/taxi | 2.8581e+08 | 2.78696e+08 | 1.02553 | ns |
| compress time/Arade | 4.44768e+08 | 4.24609e+08 | 1.04748 | ns |
| parquet_rs-zstd compress time/Arade | 2.79812e+09 | 2.7229e+09 | 1.02763 | ns |
| decompress time/Arade | 2.83391e+08 | 2.74203e+08 | 1.03351 | ns |
| parquet_rs-zstd decompress time/Arade | 6.22286e+08 | 6.15232e+08 | 1.01147 | ns |
| compress time/Bimbo | 1.60927e+09 | 1.51558e+09 | 1.06182 | ns |
| parquet_rs-zstd compress time/Bimbo | 1.87531e+10 | 1.84866e+10 | 1.01442 | ns |
| decompress time/Bimbo | 9.97075e+08 | 1.01242e+09 | 0.984848 | ns |
| parquet_rs-zstd decompress time/Bimbo | 2.35629e+09 | 2.30868e+09 | 1.02062 | ns |
| compress time/CMSprovider | 1.10527e+09 | 1.04821e+09 | 1.05443 | ns |
| parquet_rs-zstd compress time/CMSprovider | 8.33657e+09 | 8.12626e+09 | 1.02588 | ns |
| decompress time/CMSprovider | 7.10442e+08 | 7.18079e+08 | 0.989364 | ns |
| parquet_rs-zstd decompress time/CMSprovider | 1.52316e+09 | 1.53684e+09 | 0.991099 | ns |
| compress time/Euro2016 | 2.66191e+08 | 2.59354e+08 | 1.02636 | ns |
| parquet_rs-zstd compress time/Euro2016 | 1.49128e+09 | 1.47757e+09 | 1.00928 | ns |
| decompress time/Euro2016 | 2.22068e+08 | 2.24112e+08 | 0.990879 | ns |
| parquet_rs-zstd decompress time/Euro2016 | 4.10681e+08 | 4.07395e+08 | 1.00807 | ns |
| compress time/Food | 1.82191e+08 | 1.72932e+08 | 1.05354 | ns |
| parquet_rs-zstd compress time/Food | 9.95738e+08 | 9.89676e+08 | 1.00612 | ns |
| decompress time/Food | 8.97381e+07 | 9.03861e+07 | 0.992831 | ns |
| parquet_rs-zstd decompress time/Food | 1.95126e+08 | 1.97753e+08 | 0.986716 | ns |
| compress time/HashTags | 2.56495e+09 | 2.60011e+09 | 0.986479 | ns |
| parquet_rs-zstd compress time/HashTags | 2.68585e+09 | 2.81415e+09 | 0.954407 | ns |
| decompress time/HashTags | 2.52069e+09 | 2.55762e+09 | 0.98556 | ns |
| parquet_rs-zstd decompress time/HashTags | 6.32352e+08 | 6.84236e+08 | 0.924172 | ns |
| compress time/TPC-H l_comment chunked | 2.32601e+08 | 2.29915e+08 | 1.01168 | ns |
| parquet_rs-zstd compress time/TPC-H l_comment chunked | 8.6493e+08 | 8.55352e+08 | 1.0112 | ns |
| decompress time/TPC-H l_comment chunked | 1.03225e+08 | 1.00597e+08 | 1.02612 | ns |
| parquet_rs-zstd decompress time/TPC-H l_comment chunked | 2.43227e+08 | 2.29758e+08 | 1.05862 | ns |
| compress time/TPC-H l_comment canonical | 7.20519e+08 | 7.3837e+08 | 0.975824 | ns |
| parquet_rs-zstd compress time/TPC-H l_comment canonical | 8.79501e+08 | 8.80001e+08 | 0.999433 | ns |
| decompress time/TPC-H l_comment canonical | 1.06478e+08 | 1.08691e+08 | 0.979638 | ns |
| parquet_rs-zstd decompress time/TPC-H l_comment canonical | 2.43767e+08 | 2.33642e+08 | 1.04333 | ns |
| compress time/wide table cols=10 chunks=1 rows=1000 | 696371 | 541112 | 1.28693 | ns |
| parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 | 766185 | 760807 | 1.00707 | ns |
| decompress time/wide table cols=10 chunks=1 rows=1000 | 283077 | 290569 | 0.974216 | ns |
| parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 | 284232 | 272944 | 1.04136 | ns |
| compress time/wide table cols=100 chunks=1 rows=1000 | 4.81666e+06 | 4.21043e+06 | 1.14398 | ns |
| parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 | 7.33848e+06 | 7.24145e+06 | 1.0134 | ns |
| decompress time/wide table cols=100 chunks=1 rows=1000 | 2.16066e+06 | 2.20787e+06 | 0.978616 | ns |
| parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 | 2.70967e+06 | 2.70083e+06 | 1.00327 | ns |
| compress time/wide table cols=1000 chunks=1 rows=1000 | 5.06531e+07 | 5.25759e+07 | 0.963429 | ns |
| parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 | 8.61639e+07 | 8.67847e+07 | 0.992847 | ns |
| decompress time/wide table cols=1000 chunks=1 rows=1000 | 4.09939e+07 | 3.99458e+07 | 1.02624 | ns |
| parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 | 2.79594e+07 | 2.93949e+07 | 0.951165 | ns |
| compress time/wide table cols=10 chunks=50 rows=1000 | 3.0984e+06 | 2.88256e+06 | 1.07488 | ns |
| parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 | 1.09534e+06 | 1.04842e+06 | 1.04475 | ns |
| decompress time/wide table cols=10 chunks=50 rows=1000 | 259893 | 237006 | 1.09657 | ns |
| parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 | 269244 | 264411 | 1.01828 | ns |
| compress time/wide table cols=100 chunks=50 rows=1000 | 2.82999e+07 | 2.8331e+07 | 0.998902 | ns |
| parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 | 1.05934e+07 | 1.17157e+07 | 0.904208 | ns |
| decompress time/wide table cols=100 chunks=50 rows=1000 | 2.14459e+06 | 2.10946e+06 | 1.01665 | ns |
| parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 | 2.7228e+06 | 2.66766e+06 | 1.02067 | ns |
| compress time/wide table cols=1000 chunks=50 rows=1000 | 3.49824e+08 | 3.47319e+08 | 1.00721 | ns |
| parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 | 1.42171e+08 | 1.47162e+08 | 0.966085 | ns |
| decompress time/wide table cols=1000 chunks=50 rows=1000 | 4.09427e+07 | 4.14094e+07 | 0.988731 | ns |
| parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 | 2.83738e+07 | 2.95868e+07 | 0.958999 | ns |
| vortex size/taxi | 5.54821e+07 | 5.54821e+07 | 1 | bytes |
| parquet-zstd size/taxi | 5.59542e+07 | 5.59542e+07 | 1 | bytes |
| vortex:parquet-zstd size/taxi | 0.991563 | 0.991563 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/taxi | 0.121412 | 0.115912 | 1.04745 | ratio |
| vortex:parquet-zstd ratio decompress time/taxi | 0.431968 | 0.44419 | 0.972484 | ratio |
| vortex size/Arade | 1.42473e+08 | 1.42473e+08 | 1 | bytes |
| parquet-zstd size/Arade | 3.0538e+08 | 3.0538e+08 | 1 | bytes |
| vortex:parquet-zstd size/Arade | 0.466543 | 0.466543 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/Arade | 0.158952 | 0.15594 | 1.01932 | ratio |
| vortex:parquet-zstd ratio decompress time/Arade | 0.455403 | 0.445691 | 1.02179 | ratio |
| vortex size/Bimbo | 5.19886e+08 | 5.19886e+08 | 1 | bytes |
| parquet-zstd size/Bimbo | 3.89789e+08 | 3.89789e+08 | 1 | bytes |
| vortex:parquet-zstd size/Bimbo | 1.33376 | 1.33376 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/Bimbo | 0.0858137 | 0.081983 | 1.04673 | ratio |
| vortex:parquet-zstd ratio decompress time/Bimbo | 0.423154 | 0.438526 | 0.964948 | ratio |
| vortex size/CMSprovider | 5.04744e+08 | 5.04744e+08 | 1 | bytes |
| parquet-zstd size/CMSprovider | 3.84908e+08 | 3.84908e+08 | 1 | bytes |
| vortex:parquet-zstd size/CMSprovider | 1.31134 | 1.31134 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/CMSprovider | 0.132581 | 0.128991 | 1.02783 | ratio |
| vortex:parquet-zstd ratio decompress time/CMSprovider | 0.466427 | 0.467245 | 0.998249 | ratio |
| vortex size/Euro2016 | 1.71734e+08 | 1.71734e+08 | 1 | bytes |
| parquet-zstd size/Euro2016 | 1.23121e+08 | 1.23121e+08 | 1 | bytes |
| vortex:parquet-zstd size/Euro2016 | 1.39485 | 1.39485 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/Euro2016 | 0.178498 | 0.175527 | 1.01693 | ratio |
| vortex:parquet-zstd ratio decompress time/Euro2016 | 0.540731 | 0.550111 | 0.982949 | ratio |
| vortex size/Food | 4.47057e+07 | 4.47057e+07 | 1 | bytes |
| parquet-zstd size/Food | 3.62159e+07 | 3.62159e+07 | 1 | bytes |
| vortex:parquet-zstd size/Food | 1.23442 | 1.23442 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/Food | 0.182971 | 0.174736 | 1.04713 | ratio |
| vortex:parquet-zstd ratio decompress time/Food | 0.459899 | 0.457067 | 1.0062 | ratio |
| vortex size/HashTags | 2.11569e+08 | 2.11569e+08 | 1 | bytes |
| parquet-zstd size/HashTags | 1.34024e+08 | 1.34024e+08 | 1 | bytes |
| vortex:parquet-zstd size/HashTags | 1.5786 | 1.5786 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/HashTags | 0.954987 | 0.923939 | 1.0336 | ratio |
| vortex:parquet-zstd ratio decompress time/HashTags | 3.98622 | 3.73793 | 1.06643 | ratio |
| vortex size/TPC-H l_comment chunked | 8.23649e+07 | 8.32964e+07 | 0.988817 | bytes |
| parquet-zstd size/TPC-H l_comment chunked | 5.69231e+07 | 5.69172e+07 | 1.0001 | bytes |
| vortex:parquet-zstd size/TPC-H l_comment chunked | 1.44695 | 1.46347 | 0.988714 | ratio |
| vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked | 0.268924 | 0.268795 | 1.00048 | ratio |
| vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked | 0.424396 | 0.437838 | 0.969299 | ratio |
| vortex size/TPC-H l_comment canonical | 7.67594e+07 | 7.67257e+07 | 1.00044 | bytes |
| parquet-zstd size/TPC-H l_comment canonical | 5.69204e+07 | 5.69186e+07 | 1.00003 | bytes |
| vortex:parquet-zstd size/TPC-H l_comment canonical | 1.34854 | 1.34799 | 1.00041 | ratio |
| vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical | 0.819236 | 0.839056 | 0.976378 | ratio |
| vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical | 0.436802 | 0.465202 | 0.938951 | ratio |
| vortex size/wide table cols=10 chunks=1 rows=1000 | 88432 | 88432 | 1 | bytes |
| parquet-zstd size/wide table cols=10 chunks=1 rows=1000 | 93484 | 93484 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=10 chunks=1 rows=1000 | 0.945959 | 0.945959 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=10 chunks=1 rows=1000 | 0.908881 | 0.711234 | 1.27789 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=10 chunks=1 rows=1000 | 0.995936 | 1.06457 | 0.935526 | ratio |
| vortex size/wide table cols=100 chunks=1 rows=1000 | 877912 | 877912 | 1 | bytes |
| parquet-zstd size/wide table cols=100 chunks=1 rows=1000 | 934804 | 934804 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 | 0.93914 | 0.93914 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000 | 0.656356 | 0.581435 | 1.12886 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000 | 0.797389 | 0.81748 | 0.975423 | ratio |
| vortex size/wide table cols=1000 chunks=1 rows=1000 | 8.77271e+06 | 8.77271e+06 | 1 | bytes |
| parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 | 9.348e+06 | 9.348e+06 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 | 0.938458 | 0.938458 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000 | 0.58787 | 0.60582 | 0.970371 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000 | 1.46619 | 1.35894 | 1.07893 | ratio |
| vortex size/wide table cols=10 chunks=50 rows=1000 | 88432 | 88432 | 1 | bytes |
| parquet-zstd size/wide table cols=10 chunks=50 rows=1000 | 93484 | 93484 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=10 chunks=50 rows=1000 | 0.945959 | 0.945959 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=10 chunks=50 rows=1000 | 2.8287 | 2.74942 | 1.02883 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=10 chunks=50 rows=1000 | 0.965269 | 0.896355 | 1.07688 | ratio |
| vortex size/wide table cols=100 chunks=50 rows=1000 | 877912 | 877912 | 1 | bytes |
| parquet-zstd size/wide table cols=100 chunks=50 rows=1000 | 934804 | 934804 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 | 0.93914 | 0.93914 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000 | 2.67147 | 2.41822 | 1.10473 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000 | 0.787643 | 0.790756 | 0.996064 | ratio |
| vortex size/wide table cols=1000 chunks=50 rows=1000 | 8.77271e+06 | 8.77271e+06 | 1 | bytes |
| parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 | 9.348e+06 | 9.348e+06 | 1 | bytes |
| vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 | 0.938458 | 0.938458 | 1 | ratio |
| vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000 | 2.46059 | 2.36011 | 1.04257 | ratio |
| vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000 | 1.44298 | 1.39959 | 1.031 | ratio |
Benchmarks: Clickbench on NVME
Table of Results
| name | PR 5ddde6b8 | base db33b9fe | ratio (PR/base) | unit |
|---|---|---|---|---|
| clickbench_q00/DataFusion:parquet | 1596520 | 1.6237e+06 | 0.983263 | ns |
| clickbench_q01/DataFusion:parquet | 28035537 | 2.85259e+07 | 0.982811 | ns |
| clickbench_q02/DataFusion:parquet | 62411322 | 6.23615e+07 | 1.0008 | ns |
| clickbench_q03/DataFusion:parquet | 57434121 | 5.67376e+07 | 1.01228 | ns |
| clickbench_q04/DataFusion:parquet | 391159814 | 3.75815e+08 | 1.04083 | ns |
| clickbench_q05/DataFusion:parquet | 404214940 | 3.82014e+08 | 1.05812 | ns |
| clickbench_q06/DataFusion:parquet | 1698737 | 1.65399e+06 | 1.02706 | ns |
| clickbench_q07/DataFusion:parquet | 34599055 | 3.22022e+07 | 1.07443 | ns |
| clickbench_q08/DataFusion:parquet | 472465168 | 4.6582e+08 | 1.01426 | ns |
| clickbench_q09/DataFusion:parquet | 688000442 | 6.8647e+08 | 1.00223 | ns |
| clickbench_q10/DataFusion:parquet | 152489179 | 1.53229e+08 | 0.995169 | ns |
| clickbench_q11/DataFusion:parquet | 176553511 | 1.79768e+08 | 0.982119 | ns |
| clickbench_q12/DataFusion:parquet | 410240239 | 4.15972e+08 | 0.986222 | ns |
| clickbench_q13/DataFusion:parquet | 582697711 | 5.84987e+08 | 0.996087 | ns |
| clickbench_q14/DataFusion:parquet | 401475888 | 4.0587e+08 | 0.989174 | ns |
| clickbench_q15/DataFusion:parquet | 451083283 | 4.49084e+08 | 1.00445 | ns |
| clickbench_q16/DataFusion:parquet | 837297721 | 8.33061e+08 | 1.00509 | ns |
| clickbench_q17/DataFusion:parquet | 786749392 | 7.92974e+08 | 0.99215 | ns |
| clickbench_q18/DataFusion:parquet | 1595312925 | 1.54418e+09 | 1.03312 | ns |
| clickbench_q19/DataFusion:parquet | 44396191 | 4.35576e+07 | 1.01925 | ns |
| clickbench_q20/DataFusion:parquet | 663714706 | 6.55603e+08 | 1.01237 | ns |
| clickbench_q21/DataFusion:parquet | 750977308 | 7.47421e+08 | 1.00476 | ns |
| clickbench_q22/DataFusion:parquet | 1077293619 | 1.10306e+09 | 0.976638 | ns |
| clickbench_q23/DataFusion:parquet | 4921993040 | 4.83427e+09 | 1.01815 | ns |
| clickbench_q24/DataFusion:parquet | 254763167 | 2.58649e+08 | 0.984975 | ns |
| clickbench_q25/DataFusion:parquet | 220433113 | 2.21186e+08 | 0.996595 | ns |
| clickbench_q26/DataFusion:parquet | 288895951 | 2.88439e+08 | 1.00158 | ns |
| clickbench_q27/DataFusion:parquet | 1093037251 | 1.08751e+09 | 1.00508 | ns |
| clickbench_q28/DataFusion:parquet | 5928486433 | 5.94222e+09 | 0.997688 | ns |
| clickbench_q29/DataFusion:parquet | 243228273 | 2.42782e+08 | 1.00184 | ns |
| clickbench_q30/DataFusion:parquet | 395179014 | 3.87503e+08 | 1.01981 | ns |
| clickbench_q31/DataFusion:parquet | 446043459 | 4.44946e+08 | 1.00247 | ns |
| clickbench_q32/DataFusion:parquet | 1361253413 | 1.35283e+09 | 1.00622 | ns |
| clickbench_q33/DataFusion:parquet | 1677372907 | 1.67302e+09 | 1.0026 | ns |
| clickbench_q34/DataFusion:parquet | 1716982343 | 1.73816e+09 | 0.987819 | ns |
| clickbench_q35/DataFusion:parquet | 628435553 | 6.34872e+08 | 0.989862 | ns |
| clickbench_q36/DataFusion:parquet | 184938360 | 1.81058e+08 | 1.02143 | ns |
| clickbench_q37/DataFusion:parquet | 70708644 | 6.98908e+07 | 1.0117 | ns |
| clickbench_q38/DataFusion:parquet | 110911185 | 1.05449e+08 | 1.0518 | ns |
| clickbench_q39/DataFusion:parquet | 323249851 | 3.18838e+08 | 1.01384 | ns |
| clickbench_q40/DataFusion:parquet | 44346380 | 4.32389e+07 | 1.02561 | ns |
| clickbench_q41/DataFusion:parquet | 40118417 | 3.96728e+07 | 1.01123 | ns |
| clickbench_q42/DataFusion:parquet | 52043496 | 5.30613e+07 | 0.980819 | ns |
| clickbench_q00/DataFusion:vortex-file-compressed | 1804097 | 1.69117e+06 | 1.06677 | ns |
| clickbench_q01/DataFusion:vortex-file-compressed | 9086755 | 9.64179e+06 | 0.942435 | ns |
| clickbench_q02/DataFusion:vortex-file-compressed | 29431193 | 2.8479e+07 | 1.03343 | ns |
| clickbench_q03/DataFusion:vortex-file-compressed | 35898597 | 3.50673e+07 | 1.02371 | ns |
| clickbench_q04/DataFusion:vortex-file-compressed | 324017601 | 3.24231e+08 | 0.999341 | ns |
| clickbench_q05/DataFusion:vortex-file-compressed | 337989686 | 3.44844e+08 | 0.980125 | ns |
| clickbench_q06/DataFusion:vortex-file-compressed | 1736000 | 1.69304e+06 | 1.02537 | ns |
| clickbench_q07/DataFusion:vortex-file-compressed | 13228036 | 1.27474e+07 | 1.0377 | ns |
| clickbench_q08/DataFusion:vortex-file-compressed | 413713674 | 3.89057e+08 | 1.06338 | ns |
| clickbench_q09/DataFusion:vortex-file-compressed | 564574541 | 5.48964e+08 | 1.02844 | ns |
| clickbench_q10/DataFusion:vortex-file-compressed | 84381665 | 8.72182e+07 | 0.967478 | ns |
| clickbench_q11/DataFusion:vortex-file-compressed | 100072237 | 1.06868e+08 | 0.936414 | ns |
| clickbench_q12/DataFusion:vortex-file-compressed | 296670090 | 3.05165e+08 | 0.972164 | ns |
| clickbench_q13/DataFusion:vortex-file-compressed | 505894466 | 5.10326e+08 | 0.991317 | ns |
| clickbench_q14/DataFusion:vortex-file-compressed | 281087346 | 2.9631e+08 | 0.948628 | ns |
| clickbench_q15/DataFusion:vortex-file-compressed | 380857134 | 3.85955e+08 | 0.986791 | ns |
| clickbench_q16/DataFusion:vortex-file-compressed | 816228739 | 8.22289e+08 | 0.99263 | ns |
| clickbench_q17/DataFusion:vortex-file-compressed | 790238923 | 8.03357e+08 | 0.983671 | ns |
| clickbench_q18/DataFusion:vortex-file-compressed | 1511902257 | 1.51097e+09 | 1.00062 | ns |
| clickbench_q19/DataFusion:vortex-file-compressed | 17232614 | 1.82755e+07 | 0.942933 | ns |
| clickbench_q20/DataFusion:vortex-file-compressed | 343776417 | 3.58804e+08 | 0.958119 | ns |
| clickbench_q21/DataFusion:vortex-file-compressed | 409721316 | 4.14984e+08 | 0.987319 | ns |
| clickbench_q22/DataFusion:vortex-file-compressed | 820737614 | 8.28661e+08 | 0.990438 | ns |
| clickbench_q23/DataFusion:vortex-file-compressed | 3059846901 | 3.06313e+09 | 0.998927 | ns |
| clickbench_q24/DataFusion:vortex-file-compressed | 152500327 | 1.46489e+08 | 1.04103 | ns |
| clickbench_q25/DataFusion:vortex-file-compressed | 133438150 | 1.24777e+08 | 1.06942 | ns |
| clickbench_q26/DataFusion:vortex-file-compressed | 184865375 | 1.77374e+08 | 1.04223 | ns |
| clickbench_q27/DataFusion:vortex-file-compressed | 853026592 | 8.59584e+08 | 0.992371 | ns |
| clickbench_q28/DataFusion:vortex-file-compressed | 6809640271 | 6.73542e+09 | 1.01102 | ns |
| clickbench_q29/DataFusion:vortex-file-compressed | 234247023 | 2.31951e+08 | 1.0099 | ns |
| clickbench_q30/DataFusion:vortex-file-compressed | 272439648 | 2.75384e+08 | 0.989307 | ns |
| clickbench_q31/DataFusion:vortex-file-compressed | 316244839 | 3.15236e+08 | 1.0032 | ns |
| clickbench_q32/DataFusion:vortex-file-compressed | 1411637219 | 1.40367e+09 | 1.00567 | ns |
| clickbench_q33/DataFusion:vortex-file-compressed | 1629857667 | 1.62267e+09 | 1.00443 | ns |
| clickbench_q34/DataFusion:vortex-file-compressed | 1616776929 | 1.64541e+09 | 0.9826 | ns |
| clickbench_q35/DataFusion:vortex-file-compressed | 600946113 | 5.82247e+08 | 1.03212 | ns |
| clickbench_q36/DataFusion:vortex-file-compressed | 106710826 | 1.06557e+08 | 1.00144 | ns |
| clickbench_q37/DataFusion:vortex-file-compressed | 46647312 | 4.94322e+07 | 0.943663 | ns |
| clickbench_q38/DataFusion:vortex-file-compressed | 34219303 | 3.64907e+07 | 0.937754 | ns |
| clickbench_q39/DataFusion:vortex-file-compressed | 217751804 | 2.21192e+08 | 0.984449 | ns |
| clickbench_q40/DataFusion:vortex-file-compressed | 21491165 | 2.30877e+07 | 0.930848 | ns |
| clickbench_q41/DataFusion:vortex-file-compressed | 18714500 | 1.99636e+07 | 0.937433 | ns |
| clickbench_q42/DataFusion:vortex-file-compressed | 32685655 | 3.36638e+07 | 0.970944 | ns |
| clickbench_q00/DuckDB:parquet | 129754325 | 1.32578e+08 | 0.978702 | ns |
| clickbench_q01/DuckDB:parquet | 54117904 | 5.54818e+07 | 0.975418 | ns |
| clickbench_q02/DuckDB:parquet | 69947821 | 7.08547e+07 | 0.9872 | ns |
| clickbench_q03/DuckDB:parquet | 67888873 | 6.96942e+07 | 0.974096 | ns |
| clickbench_q04/DuckDB:parquet | 213918784 | 2.15525e+08 | 0.992549 | ns |
| clickbench_q05/DuckDB:parquet | 241752228 | 2.44049e+08 | 0.990589 | ns |
| clickbench_q06/DuckDB:parquet | 73606413 | 7.4289e+07 | 0.990812 | ns |
| clickbench_q07/DuckDB:parquet | 64134439 | 6.64946e+07 | 0.964505 | ns |
| clickbench_q08/DuckDB:parquet | 279961990 | 2.81811e+08 | 0.993439 | ns |
| clickbench_q09/DuckDB:parquet | 411349516 | 4.12171e+08 | 0.998007 | ns |
| clickbench_q10/DuckDB:parquet | 111072748 | 1.11743e+08 | 0.994004 | ns |
| clickbench_q11/DuckDB:parquet | 123009262 | 1.24947e+08 | 0.984493 | ns |
| clickbench_q12/DuckDB:parquet | 258547118 | 2.59201e+08 | 0.997476 | ns |
| clickbench_q13/DuckDB:parquet | 427460366 | 4.23127e+08 | 1.01024 | ns |
| clickbench_q14/DuckDB:parquet | 293121541 | 2.88237e+08 | 1.01695 | ns |
| clickbench_q15/DuckDB:parquet | 247826969 | 2.48153e+08 | 0.998685 | ns |
| clickbench_q16/DuckDB:parquet | 583836217 | 5.84815e+08 | 0.998326 | ns |
| clickbench_q17/DuckDB:parquet | 502705894 | 5.08037e+08 | 0.989507 | ns |
| clickbench_q18/DuckDB:parquet | 979779468 | 9.79607e+08 | 1.00018 | ns |
| clickbench_q19/DuckDB:parquet | 65440135 | 6.51312e+07 | 1.00474 | ns |
| clickbench_q20/DuckDB:parquet | 397703076 | 3.78245e+08 | 1.05144 | ns |
| clickbench_q21/DuckDB:parquet | 493610780 | 5.02055e+08 | 0.983181 | ns |
| clickbench_q22/DuckDB:parquet | 880871376 | 8.77135e+08 | 1.00426 | ns |
| clickbench_q23/DuckDB:parquet | 290512414 | 2.85398e+08 | 1.01792 | ns |
| clickbench_q24/DuckDB:parquet | 95369045 | 1.09058e+08 | 0.874482 | ns |
| clickbench_q25/DuckDB:parquet | 139253831 | 1.43296e+08 | 0.971793 | ns |
| clickbench_q26/DuckDB:parquet | 63865275 | 6.26894e+07 | 1.01876 | ns |
| clickbench_q27/DuckDB:parquet | 631792051 | 6.4231e+08 | 0.983625 | ns |
| clickbench_q28/DuckDB:parquet | 5748698653 | 5.79433e+09 | 0.992125 | ns |
| clickbench_q29/DuckDB:parquet | 74128672 | 7.48726e+07 | 0.990065 | ns |
| clickbench_q30/DuckDB:parquet | 295651606 | 2.97241e+08 | 0.994654 | ns |
| clickbench_q31/DuckDB:parquet | 342459779 | 3.44858e+08 | 0.993045 | ns |
| clickbench_q32/DuckDB:parquet | 1041589131 | 1.03756e+09 | 1.00388 | ns |
| clickbench_q33/DuckDB:parquet | 1072730232 | 1.06088e+09 | 1.01117 | ns |
| clickbench_q34/DuckDB:parquet | 1136063251 | 1.14301e+09 | 0.993924 | ns |
| clickbench_q35/DuckDB:parquet | 379080047 | 3.75585e+08 | 1.00931 | ns |
| clickbench_q36/DuckDB:parquet | 68737639 | 6.94709e+07 | 0.989445 | ns |
| clickbench_q37/DuckDB:parquet | 62777563 | 6.29883e+07 | 0.996655 | ns |
| clickbench_q38/DuckDB:parquet | 67773094 | 6.80503e+07 | 0.995926 | ns |
| clickbench_q39/DuckDB:parquet | 100780116 | 1.03478e+08 | 0.973931 | ns |
| clickbench_q40/DuckDB:parquet | 64982757 | 6.58316e+07 | 0.987106 | ns |
| clickbench_q41/DuckDB:parquet | 73801187 | 7.63314e+07 | 0.966852 | ns |
| clickbench_q42/DuckDB:parquet | 54563058 | 5.95232e+07 | 0.916669 | ns |
| clickbench_q00/DuckDB:vortex-file-compressed | 9406816 | 9.518e+06 | 0.988318 | ns |
| clickbench_q01/DuckDB:vortex-file-compressed | 12924957 | 1.35628e+07 | 0.952974 | ns |
| clickbench_q02/DuckDB:vortex-file-compressed | 28772118 | 2.79182e+07 | 1.03058 | ns |
| clickbench_q03/DuckDB:vortex-file-compressed | 34056412 | 3.3709e+07 | 1.01031 | ns |
| clickbench_q04/DuckDB:vortex-file-compressed | 210499856 | 2.1029e+08 | 1.001 | ns |
| clickbench_q05/DuckDB:vortex-file-compressed | 160877569 | 1.62789e+08 | 0.988259 | ns |
| clickbench_q06/DuckDB:vortex-file-compressed | 37790248 | 3.80212e+07 | 0.993925 | ns |
| clickbench_q07/DuckDB:vortex-file-compressed | 18344078 | 2.04226e+07 | 0.898225 | ns |
| clickbench_q08/DuckDB:vortex-file-compressed | 278550652 | 2.79536e+08 | 0.996476 | ns |
| clickbench_q09/DuckDB:vortex-file-compressed | 395598387 | 3.96568e+08 | 0.997555 | ns |
| clickbench_q10/DuckDB:vortex-file-compressed | 90506598 | 9.0407e+07 | 1.0011 | ns |
| clickbench_q11/DuckDB:vortex-file-compressed | 99306767 | 1.00399e+08 | 0.989123 | ns |
| clickbench_q12/DuckDB:vortex-file-compressed | 183776087 | 1.84326e+08 | 0.997016 | ns |
| clickbench_q13/DuckDB:vortex-file-compressed | 361576336 | 3.63863e+08 | 0.993716 | ns |
| clickbench_q14/DuckDB:vortex-file-compressed | 208544711 | 2.08866e+08 | 0.998462 | ns |
| clickbench_q15/DuckDB:vortex-file-compressed | 243328072 | 2.40182e+08 | 1.0131 | ns |
| clickbench_q16/DuckDB:vortex-file-compressed | 521983982 | 5.2042e+08 | 1.00301 | ns |
| clickbench_q17/DuckDB:vortex-file-compressed | 438478260 | 4.358e+08 | 1.00615 | ns |
| clickbench_q18/DuckDB:vortex-file-compressed | 905992422 | 9.0611e+08 | 0.99987 | ns |
| clickbench_q19/DuckDB:vortex-file-compressed | 217127903 | 2.2076e+08 | 0.983546 | ns |
| clickbench_q20/DuckDB:vortex-file-compressed | 290847939 | 2.93823e+08 | 0.989876 | ns |
| clickbench_q21/DuckDB:vortex-file-compressed | 369939004 | 3.55799e+08 | 1.03974 | ns |
| clickbench_q22/DuckDB:vortex-file-compressed | 578316349 | 6.0076e+08 | 0.962641 | ns |
| clickbench_q23/DuckDB:vortex-file-compressed | 575936296 | 5.75533e+08 | 1.0007 | ns |
| clickbench_q24/DuckDB:vortex-file-compressed | 26943150 | 2.51886e+07 | 1.06966 | ns |
| clickbench_q25/DuckDB:vortex-file-compressed | 84377283 | 8.51154e+07 | 0.991328 | ns |
| clickbench_q26/DuckDB:vortex-file-compressed | 26145747 | 2.6391e+07 | 0.990707 | ns |
| clickbench_q27/DuckDB:vortex-file-compressed | 558123131 | 5.66022e+08 | 0.986045 | ns |
| clickbench_q28/DuckDB:vortex-file-compressed | 5627888462 | 5.65437e+09 | 0.995317 | ns |
| clickbench_q29/DuckDB:vortex-file-compressed | 32186025 | 3.20287e+07 | 1.00491 | ns |
| clickbench_q30/DuckDB:vortex-file-compressed | 191771702 | 1.94986e+08 | 0.983517 | ns |
| clickbench_q31/DuckDB:vortex-file-compressed | 255015290 | 2.57575e+08 | 0.990062 | ns |
| clickbench_q32/DuckDB:vortex-file-compressed | 1049064258 | 1.03525e+09 | 1.01335 | ns |
| clickbench_q33/DuckDB:vortex-file-compressed | 940456091 | 9.67302e+08 | 0.972247 | ns |
| clickbench_q34/DuckDB:vortex-file-compressed | 1061846320 | 1.06207e+09 | 0.999786 | ns |
| clickbench_q35/DuckDB:vortex-file-compressed | 381436575 | 3.89119e+08 | 0.980258 | ns |
| clickbench_q36/DuckDB:vortex-file-compressed | 35594223 | 3.56465e+07 | 0.998533 | ns |
| clickbench_q37/DuckDB:vortex-file-compressed | 22886959 | 2.43369e+07 | 0.940423 | ns |
| clickbench_q38/DuckDB:vortex-file-compressed | 28585075 | 3.07058e+07 | 0.930934 | ns |
| clickbench_q39/DuckDB:vortex-file-compressed | 59605854 | 5.92746e+07 | 1.00559 | ns |
| clickbench_q40/DuckDB:vortex-file-compressed | 18191330 | 1.78453e+07 | 1.01939 | ns |
| clickbench_q41/DuckDB:vortex-file-compressed | 18213362 | 1.91933e+07 | 0.948946 | ns |
| clickbench_q42/DuckDB:vortex-file-compressed | 23581301 | 2.03948e+07 | 1.15624 | ns |
| clickbench_q00/DuckDB:duckdb | 15422541 | 1.58515e+07 | 0.972939 | ns |
| clickbench_q01/DuckDB:duckdb | 5032062 | 5.55045e+06 | 0.906604 | ns |
| clickbench_q02/DuckDB:duckdb | 14550149 | 1.69297e+07 | 0.859444 | ns |
| clickbench_q03/DuckDB:duckdb | 19510877 | 2.04192e+07 | 0.955514 | ns |
| clickbench_q04/DuckDB:duckdb | 208049948 | 2.13837e+08 | 0.972938 | ns |
| clickbench_q05/DuckDB:duckdb | 248675755 | 2.42874e+08 | 1.02389 | ns |
| clickbench_q06/DuckDB:duckdb | 14021318 | 1.73975e+07 | 0.805939 | ns |
| clickbench_q07/DuckDB:duckdb | 5993477 | 6.50068e+06 | 0.921977 | ns |
| clickbench_q08/DuckDB:duckdb | 255024306 | 2.55798e+08 | 0.996975 | ns |
| clickbench_q09/DuckDB:duckdb | 358356602 | 3.57414e+08 | 1.00264 | ns |
| clickbench_q10/DuckDB:duckdb | 76293598 | 7.97504e+07 | 0.956655 | ns |
| clickbench_q11/DuckDB:duckdb | 84173399 | 8.60211e+07 | 0.97852 | ns |
| clickbench_q12/DuckDB:duckdb | 207514127 | 2.0606e+08 | 1.00706 | ns |
| clickbench_q13/DuckDB:duckdb | 353241472 | 3.51284e+08 | 1.00557 | ns |
| clickbench_q14/DuckDB:duckdb | 222591730 | 2.18813e+08 | 1.01727 | ns |
| clickbench_q15/DuckDB:duckdb | 227552101 | 2.27552e+08 | 1 | ns |
| clickbench_q16/DuckDB:duckdb | 514496637 | 5.15542e+08 | 0.997973 | ns |
| clickbench_q17/DuckDB:duckdb | 429369120 | 4.28682e+08 | 1.0016 | ns |
| clickbench_q18/DuckDB:duckdb | 851713159 | 8.48303e+08 | 1.00402 | ns |
| clickbench_q19/DuckDB:duckdb | 4747875 | 2.39649e+06 | 1.98118 | ns |
| clickbench_q20/DuckDB:duckdb | 297783214 | 2.9538e+08 | 1.00814 | ns |
| clickbench_q21/DuckDB:duckdb | 256135454 | 2.55458e+08 | 1.00265 | ns |
| clickbench_q22/DuckDB:duckdb | 391928206 | 3.91204e+08 | 1.00185 | ns |
| clickbench_q23/DuckDB:duckdb | 47307675 | 4.42704e+07 | 1.06861 | ns |
| clickbench_q24/DuckDB:duckdb | 15324094 | 1.53926e+07 | 0.99555 | ns |
| clickbench_q25/DuckDB:duckdb | 78231107 | 7.8358e+07 | 0.99838 | ns |
| clickbench_q26/DuckDB:duckdb | 13702339 | 1.41457e+07 | 0.968655 | ns |
| clickbench_q27/DuckDB:duckdb | 386625217 | 3.87004e+08 | 0.999021 | ns |
| clickbench_q28/DuckDB:duckdb | 5582008671 | 5.61012e+09 | 0.99499 | ns |
| clickbench_q29/DuckDB:duckdb | 22637784 | 2.0902e+07 | 1.08304 | ns |
| clickbench_q30/DuckDB:duckdb | 198506215 | 1.97521e+08 | 1.00499 | ns |
| clickbench_q31/DuckDB:duckdb | 237329916 | 2.41715e+08 | 0.981857 | ns |
| clickbench_q32/DuckDB:duckdb | 973149433 | 9.9469e+08 | 0.978344 | ns |
| clickbench_q33/DuckDB:duckdb | 870557753 | 8.7666e+08 | 0.99304 | ns |
| clickbench_q34/DuckDB:duckdb | 949506027 | 9.4651e+08 | 1.00317 | ns |
| clickbench_q35/DuckDB:duckdb | 284817315 | 2.84557e+08 | 1.00091 | ns |
| clickbench_q36/DuckDB:duckdb | 24591010 | 3.082e+07 | 0.797892 | ns |
| clickbench_q37/DuckDB:duckdb | 4766220 | 6.39314e+06 | 0.745521 | ns |
| clickbench_q38/DuckDB:duckdb | 7398908 | 8.33008e+06 | 0.888216 | ns |
| clickbench_q39/DuckDB:duckdb | 44966629 | 4.87766e+07 | 0.92189 | ns |
| clickbench_q40/DuckDB:duckdb | 3717981 | 4.06738e+06 | 0.914098 | ns |
| clickbench_q41/DuckDB:duckdb | 5259314 | 5.23345e+06 | 1.00494 | ns |
| clickbench_q42/DuckDB:duckdb | 8031946 | 8.14927e+06 | 0.985603 | ns |
Yes, didn't have the row_range from the scan builder included. Rerunning the benchmarks
Benchmarks: TPC-H on NVME
Table of Results
| name | PR 5ddde6b8 | base db33b9fe | ratio (PR/base) | unit |
|---|---|---|---|---|
| tpch_q01/DataFusion:arrow | 58803984 | 6.17618e+07 | 0.952109 | ns |
| tpch_q02/DataFusion:arrow | 16761310 | 1.70195e+07 | 0.984829 | ns |
| tpch_q03/DataFusion:arrow | 22144341 | 2.19103e+07 | 1.01068 | ns |
| tpch_q04/DataFusion:arrow | 12007407 | 1.19697e+07 | 1.00315 | ns |
| tpch_q05/DataFusion:arrow | 31192519 | 3.17092e+07 | 0.983707 | ns |
| tpch_q06/DataFusion:arrow | 7110393 | 7.48295e+06 | 0.950213 | ns |
| tpch_q07/DataFusion:arrow | 54604445 | 5.53685e+07 | 0.9862 | ns |
| tpch_q08/DataFusion:arrow | 17856136 | 1.84131e+07 | 0.969749 | ns |
| tpch_q09/DataFusion:arrow | 32397084 | 3.2908e+07 | 0.984473 | ns |
| tpch_q10/DataFusion:arrow | 29233179 | 2.95254e+07 | 0.990103 | ns |
| tpch_q11/DataFusion:arrow | 7698092 | 7.77654e+06 | 0.989913 | ns |
| tpch_q12/DataFusion:arrow | 24037709 | 2.4735e+07 | 0.971808 | ns |
| tpch_q13/DataFusion:arrow | 14740128 | 1.53758e+07 | 0.958658 | ns |
| tpch_q14/DataFusion:arrow | 6797537 | 6.8882e+06 | 0.986838 | ns |
| tpch_q15/DataFusion:arrow | 13908021 | 1.3809e+07 | 1.00717 | ns |
| tpch_q16/DataFusion:arrow | 14961548 | 1.52689e+07 | 0.979868 | ns |
| tpch_q17/DataFusion:arrow | 52502045 | 4.89934e+07 | 1.07162 | ns |
| tpch_q18/DataFusion:arrow | 111299730 | 1.10727e+08 | 1.00517 | ns |
| tpch_q19/DataFusion:arrow | 18647904 | 2.28251e+07 | 0.816991 | ns |
| tpch_q20/DataFusion:arrow | 20706563 | 2.14791e+07 | 0.964035 | ns |
| tpch_q21/DataFusion:arrow | 79867608 | 8.05018e+07 | 0.992123 | ns |
| tpch_q22/DataFusion:arrow | 10980449 | 1.04303e+07 | 1.05275 | ns |
| tpch_q01/DataFusion:parquet | 193990068 | 1.57457e+08 | 1.23202 | ns |
| tpch_q02/DataFusion:parquet | 81084613 | 7.67535e+07 | 1.05643 | ns |
| tpch_q03/DataFusion:parquet | 87151867 | 7.98339e+07 | 1.09167 | ns |
| tpch_q04/DataFusion:parquet | 64406204 | 5.86142e+07 | 1.09882 | ns |
| tpch_q05/DataFusion:parquet | 99121123 | 9.66861e+07 | 1.02518 | ns |
| tpch_q06/DataFusion:parquet | 35445053 | 4.81353e+07 | 0.736363 | ns |
| tpch_q07/DataFusion:parquet | 113762162 | 1.19145e+08 | 0.954822 | ns |
| tpch_q08/DataFusion:parquet | 92873128 | 9.14737e+07 | 1.0153 | ns |
| tpch_q09/DataFusion:parquet | 138666595 | 1.36033e+08 | 1.01936 | ns |
| tpch_q10/DataFusion:parquet | 116143057 | 1.25766e+08 | 0.923484 | ns |
| tpch_q11/DataFusion:parquet | 69521028 | 6.50173e+07 | 1.06927 | ns |
| tpch_q12/DataFusion:parquet | 102026176 | 8.41036e+07 | 1.2131 | ns |
| tpch_q13/DataFusion:parquet | 169370667 | 1.68331e+08 | 1.00618 | ns |
| tpch_q14/DataFusion:parquet | 44077210 | 4.31982e+07 | 1.02035 | ns |
| tpch_q15/DataFusion:parquet | 64252475 | 6.28039e+07 | 1.02306 | ns |
| tpch_q16/DataFusion:parquet | 47654662 | 4.78376e+07 | 0.996176 | ns |
| tpch_q17/DataFusion:parquet | 132293142 | 1.42132e+08 | 0.930778 | ns |
| tpch_q18/DataFusion:parquet | 175450064 | 1.77567e+08 | 0.988079 | ns |
| tpch_q19/DataFusion:parquet | 80071898 | 8.19764e+07 | 0.976768 | ns |
| tpch_q20/DataFusion:parquet | 76097024 | 7.19778e+07 | 1.05723 | ns |
| tpch_q21/DataFusion:parquet | 146118601 | 1.49565e+08 | 0.976955 | ns |
| tpch_q22/DataFusion:parquet | 73547294 | 7.30155e+07 | 1.00728 | ns |
| tpch_q01/DataFusion:vortex-file-compressed | 54850230 | 5.47281e+07 | 1.00223 | ns |
| tpch_q02/DataFusion:vortex-file-compressed | 30978871 | 3.25217e+07 | 0.95256 | ns |
| tpch_q03/DataFusion:vortex-file-compressed | 20401373 | 2.15384e+07 | 0.947208 | ns |
| tpch_q04/DataFusion:vortex-file-compressed | 16261687 | 1.59736e+07 | 1.01804 | ns |
| tpch_q05/DataFusion:vortex-file-compressed | 47830082 | 4.82069e+07 | 0.992184 | ns |
| tpch_q06/DataFusion:vortex-file-compressed | 6966080 | 8.0281e+06 | 0.867712 | ns |
| tpch_q07/DataFusion:vortex-file-compressed | 70232646 | 7.0631e+07 | 0.99436 | ns |
| tpch_q08/DataFusion:vortex-file-compressed | 34861594 | 3.71367e+07 | 0.938736 | ns |
| tpch_q09/DataFusion:vortex-file-compressed | 50981170 | 5.1615e+07 | 0.987721 | ns |
| tpch_q10/DataFusion:vortex-file-compressed | 40341375 | 4.13207e+07 | 0.976299 | ns |
| tpch_q11/DataFusion:vortex-file-compressed | 19559607 | 1.97292e+07 | 0.991402 | ns |
| tpch_q12/DataFusion:vortex-file-compressed | 16581108 | 1.68672e+07 | 0.983038 | ns |
| tpch_q13/DataFusion:vortex-file-compressed | 20163104 | 2.1044e+07 | 0.958141 | ns |
| tpch_q14/DataFusion:vortex-file-compressed | 10165153 | 1.0373e+07 | 0.979963 | ns |
| tpch_q15/DataFusion:vortex-file-compressed | 17590635 | 1.80607e+07 | 0.973971 | ns |
| tpch_q16/DataFusion:vortex-file-compressed | 22771536 | 2.33953e+07 | 0.973339 | ns |
| tpch_q17/DataFusion:vortex-file-compressed | 58572220 | 5.87265e+07 | 0.997373 | ns |
| tpch_q18/DataFusion:vortex-file-compressed | 90849521 | 9.48745e+07 | 0.957575 | ns |
| tpch_q19/DataFusion:vortex-file-compressed | 14501809 | 1.48829e+07 | 0.974393 | ns |
| tpch_q20/DataFusion:vortex-file-compressed | 23249297 | 2.36306e+07 | 0.983864 | ns |
| tpch_q21/DataFusion:vortex-file-compressed | 70411212 | 7.13541e+07 | 0.986786 | ns |
| tpch_q22/DataFusion:vortex-file-compressed | 15330954 | 1.55872e+07 | 0.983557 | ns |
| tpch_q01/DuckDB:parquet | 100943354 | 1.01423e+08 | 0.995269 | ns |
| tpch_q02/DuckDB:parquet | 43715024 | 4.3937e+07 | 0.994947 | ns |
| tpch_q03/DuckDB:parquet | 73511747 | 7.20295e+07 | 1.02058 | ns |
| tpch_q04/DuckDB:parquet | 56071769 | 5.6545e+07 | 0.99163 | ns |
| tpch_q05/DuckDB:parquet | 68489842 | 6.89511e+07 | 0.993311 | ns |
| tpch_q06/DuckDB:parquet | 20332908 | 2.02334e+07 | 1.00492 | ns |
| tpch_q07/DuckDB:parquet | 75921266 | 7.82579e+07 | 0.970142 | ns |
| tpch_q08/DuckDB:parquet | 89515796 | 9.0265e+07 | 0.991699 | ns |
| tpch_q09/DuckDB:parquet | 141865453 | 1.33971e+08 | 1.05893 | ns |
| tpch_q10/DuckDB:parquet | 124041263 | 1.24024e+08 | 1.00014 | ns |
| tpch_q11/DuckDB:parquet | 22920640 | 2.48456e+07 | 0.922523 | ns |
| tpch_q12/DuckDB:parquet | 57322646 | 5.76632e+07 | 0.994093 | ns |
| tpch_q13/DuckDB:parquet | 243992229 | 2.45217e+08 | 0.995004 | ns |
| tpch_q14/DuckDB:parquet | 45243351 | 4.47876e+07 | 1.01018 | ns |
| tpch_q15/DuckDB:parquet | 26391472 | 2.59742e+07 | 1.01607 | ns |
| tpch_q16/DuckDB:parquet | 48780872 | 4.971e+07 | 0.981308 | ns |
| tpch_q17/DuckDB:parquet | 55091058 | 5.57841e+07 | 0.987576 | ns |
| tpch_q18/DuckDB:parquet | 118213041 | 1.16568e+08 | 1.01411 | ns |
| tpch_q19/DuckDB:parquet | 68067674 | 6.9386e+07 | 0.981001 | ns |
| tpch_q20/DuckDB:parquet | 59022223 | 6.07591e+07 | 0.971414 | ns |
| tpch_q21/DuckDB:parquet | 173177470 | 1.81386e+08 | 0.954745 | ns |
| tpch_q22/DuckDB:parquet | 49443406 | 5.09683e+07 | 0.970081 | ns |
| tpch_q01/DuckDB:vortex-file-compressed | 31945279 | 3.34869e+07 | 0.953963 | ns |
| tpch_q02/DuckDB:vortex-file-compressed | 24189508 | 2.56259e+07 | 0.943946 | ns |
| tpch_q03/DuckDB:vortex-file-compressed | 28710537 | 2.83927e+07 | 1.01119 | ns |
| tpch_q04/DuckDB:vortex-file-compressed | 27049146 | 2.80418e+07 | 0.9646 | ns |
| tpch_q05/DuckDB:vortex-file-compressed | 36974230 | 3.89448e+07 | 0.949401 | ns |
| tpch_q06/DuckDB:vortex-file-compressed | 7771620 | 7.95142e+06 | 0.977388 | ns |
| tpch_q07/DuckDB:vortex-file-compressed | 34106608 | 3.39505e+07 | 1.0046 | ns |
| tpch_q08/DuckDB:vortex-file-compressed | 43349387 | 4.38689e+07 | 0.988157 | ns |
| tpch_q09/DuckDB:vortex-file-compressed | 58958734 | 6.0928e+07 | 0.96768 | ns |
| tpch_q10/DuckDB:vortex-file-compressed | 39715861 | 3.81438e+07 | 1.04121 | ns |
| tpch_q11/DuckDB:vortex-file-compressed | 13374419 | 1.41296e+07 | 0.946554 | ns |
| tpch_q12/DuckDB:vortex-file-compressed | 28350047 | 2.8533e+07 | 0.993589 | ns |
| tpch_q13/DuckDB:vortex-file-compressed | 38826430 | 3.85249e+07 | 1.00783 | ns |
| tpch_q14/DuckDB:vortex-file-compressed | 19012502 | 1.8681e+07 | 1.01775 | ns |
| tpch_q15/DuckDB:vortex-file-compressed | 12536124 | 1.20659e+07 | 1.03897 | ns |
| tpch_q16/DuckDB:vortex-file-compressed | 22358068 | 2.31564e+07 | 0.965523 | ns |
| tpch_q17/DuckDB:vortex-file-compressed | 59731870 | 6.05248e+07 | 0.986899 | ns |
| tpch_q18/DuckDB:vortex-file-compressed | 49651723 | 4.9301e+07 | 1.00711 | ns |
| tpch_q19/DuckDB:vortex-file-compressed | 23051050 | 2.2875e+07 | 1.0077 | ns |
| tpch_q20/DuckDB:vortex-file-compressed | 29536110 | 2.841e+07 | 1.03964 | ns |
| tpch_q21/DuckDB:vortex-file-compressed | 107173888 | 1.06795e+08 | 1.00354 | ns |
| tpch_q22/DuckDB:vortex-file-compressed | 21083180 | 2.14287e+07 | 0.983876 | ns |
| tpch_q01/DuckDB:duckdb | 14325479 | 1.42487e+07 | 1.00539 | ns |
| tpch_q02/DuckDB:duckdb | 12193803 | 1.05166e+07 | 1.15949 | ns |
| tpch_q03/DuckDB:duckdb | 14830983 | 1.45959e+07 | 1.0161 | ns |
| tpch_q04/DuckDB:duckdb | 16784307 | 1.67135e+07 | 1.00424 | ns |
| tpch_q05/DuckDB:duckdb | 15845035 | 1.66864e+07 | 0.949576 | ns |
| tpch_q06/DuckDB:duckdb | 3426526 | 3.43235e+06 | 0.998304 | ns |
| tpch_q07/DuckDB:duckdb | 17477419 | 1.91091e+07 | 0.914613 | ns |
| tpch_q08/DuckDB:duckdb | 18165537 | 1.81333e+07 | 1.00178 | ns |
| tpch_q09/DuckDB:duckdb | 48795879 | 5.30547e+07 | 0.919727 | ns |
| tpch_q10/DuckDB:duckdb | 40699588 | 4.05215e+07 | 1.0044 | ns |
| tpch_q11/DuckDB:duckdb | 6993898 | 6.70686e+06 | 1.0428 | ns |
| tpch_q12/DuckDB:duckdb | 12735084 | 1.27566e+07 | 0.998312 | ns |
| tpch_q13/DuckDB:duckdb | 46486911 | 4.2827e+07 | 1.08546 | ns |
| tpch_q14/DuckDB:duckdb | 12564297 | 1.23924e+07 | 1.01387 | ns |
| tpch_q15/DuckDB:duckdb | 7574183 | 7.45079e+06 | 1.01656 | ns |
| tpch_q16/DuckDB:duckdb | 21298820 | 1.95486e+07 | 1.08953 | ns |
| tpch_q17/DuckDB:duckdb | 14474214 | 1.45017e+07 | 0.998106 | ns |
| tpch_q18/DuckDB:duckdb | 34825582 | 3.51057e+07 | 0.99202 | ns |
| tpch_q19/DuckDB:duckdb | 20207850 | 2.06594e+07 | 0.978144 | ns |
| tpch_q20/DuckDB:duckdb | 16338966 | 1.67182e+07 | 0.977313 | ns |
| tpch_q21/DuckDB:duckdb | 50586861 | 5.02087e+07 | 1.00753 | ns |
| tpch_q22/DuckDB:duckdb | 23820934 | 2.38303e+07 | 0.999609 | ns |
Benchmarks: TPC-H on S3
Table of Results
| name | PR 5ddde6b8 | base db33b9fe | ratio (PR/base) | unit |
|---|---|---|---|---|
| tpch_q01/DataFusion:parquet | 395109845 | 3.91461e+08 | 1.00932 | ns |
| tpch_q02/DataFusion:parquet | 764523019 | 7.69606e+08 | 0.993396 | ns |
| tpch_q03/DataFusion:parquet | 559241131 | 5.53365e+08 | 1.01062 | ns |
| tpch_q04/DataFusion:parquet | 290626254 | 2.90109e+08 | 1.00178 | ns |
| tpch_q05/DataFusion:parquet | 746645471 | 7.53451e+08 | 0.990967 | ns |
| tpch_q06/DataFusion:parquet | 260844161 | 2.6148e+08 | 0.997569 | ns |
| tpch_q07/DataFusion:parquet | 726253425 | 7.99164e+08 | 0.908767 | ns |
| tpch_q08/DataFusion:parquet | 832405735 | 8.02754e+08 | 1.03694 | ns |
| tpch_q09/DataFusion:parquet | 695031095 | 6.94976e+08 | 1.00008 | ns |
| tpch_q10/DataFusion:parquet | 749158953 | 7.30788e+08 | 1.02514 | ns |
| tpch_q11/DataFusion:parquet | 548453748 | 5.20919e+08 | 1.05286 | ns |
| tpch_q12/DataFusion:parquet | 390894749 | 3.94534e+08 | 0.990777 | ns |
| tpch_q13/DataFusion:parquet | 507581602 | 4.99317e+08 | 1.01655 | ns |
| tpch_q14/DataFusion:parquet | 354322549 | 3.54479e+08 | 0.999558 | ns |
| tpch_q15/DataFusion:parquet | 570348574 | 5.76144e+08 | 0.989941 | ns |
| tpch_q16/DataFusion:parquet | 371878471 | 3.51407e+08 | 1.05826 | ns |
| tpch_q17/DataFusion:parquet | 604103611 | 5.74811e+08 | 1.05096 | ns |
| tpch_q18/DataFusion:parquet | 709869908 | 6.88701e+08 | 1.03074 | ns |
| tpch_q19/DataFusion:parquet | 445551330 | 4.48274e+08 | 0.993926 | ns |
| tpch_q20/DataFusion:parquet | 528962484 | 5.49582e+08 | 0.962481 | ns |
| tpch_q21/DataFusion:parquet | 899381081 | 8.24486e+08 | 1.09084 | ns |
| tpch_q22/DataFusion:parquet | 388929828 | 3.36933e+08 | 1.15432 | ns |
| tpch_q01/DataFusion:vortex-file-compressed | 161985140 | 1.57e+08 | 1.03175 | ns |
| tpch_q02/DataFusion:vortex-file-compressed | 128500789 | 1.24592e+08 | 1.03137 | ns |
| tpch_q03/DataFusion:vortex-file-compressed | 222273721 | 2.07906e+08 | 1.06911 | ns |
| tpch_q04/DataFusion:vortex-file-compressed | 136711761 | 1.33707e+08 | 1.02247 | ns |
| tpch_q05/DataFusion:vortex-file-compressed | 216786964 | 2.11818e+08 | 1.02346 | ns |
| tpch_q06/DataFusion:vortex-file-compressed | 90653709 | 1.05169e+08 | 0.861985 | ns |
| tpch_q07/DataFusion:vortex-file-compressed | 272685003 | 2.58916e+08 | 1.05318 | ns |
| tpch_q08/DataFusion:vortex-file-compressed | 261386573 | 2.4191e+08 | 1.08051 | ns |
| tpch_q09/DataFusion:vortex-file-compressed | 279391838 | 2.59588e+08 | 1.07629 | ns |
| tpch_q10/DataFusion:vortex-file-compressed | 249556852 | 2.64231e+08 | 0.944464 | ns |
| tpch_q11/DataFusion:vortex-file-compressed | 121893545 | 1.29001e+08 | 0.944907 | ns |
| tpch_q12/DataFusion:vortex-file-compressed | 165650779 | 1.95238e+08 | 0.848456 | ns |
| tpch_q13/DataFusion:vortex-file-compressed | 138702073 | 1.27131e+08 | 1.09101 | ns |
| tpch_q14/DataFusion:vortex-file-compressed | 129589840 | 1.30316e+08 | 0.994424 | ns |
| tpch_q15/DataFusion:vortex-file-compressed | 217762298 | 2.19247e+08 | 0.993226 | ns |
| tpch_q16/DataFusion:vortex-file-compressed | 86043442 | 7.81849e+07 | 1.10051 | ns |
| tpch_q17/DataFusion:vortex-file-compressed | 218634457 | 2.13892e+08 | 1.02217 | ns |
| tpch_q18/DataFusion:vortex-file-compressed | 298286339 | 2.60275e+08 | 1.14604 | ns |
| tpch_q19/DataFusion:vortex-file-compressed | 130839090 | 1.29073e+08 | 1.01368 | ns |
| tpch_q20/DataFusion:vortex-file-compressed | 198216752 | 1.90808e+08 | 1.03883 | ns |
| tpch_q21/DataFusion:vortex-file-compressed | 374212954 | 3.62733e+08 | 1.03165 | ns |
| tpch_q22/DataFusion:vortex-file-compressed | 107359281 | 1.07818e+08 | 0.995747 | ns |
| tpch_q01/DuckDB:parquet | 147608155 | 1.37988e+08 | 1.06972 | ns |
| tpch_q02/DuckDB:parquet | 402606921 | 3.22459e+08 | 1.24855 | ns |
| tpch_q03/DuckDB:parquet | 164598914 | 1.94988e+08 | 0.84415 | ns |
| tpch_q04/DuckDB:parquet | 106822547 | 1.28418e+08 | 0.831838 | ns |
| tpch_q05/DuckDB:parquet | 311585432 | 2.00425e+08 | 1.55462 | ns |
| tpch_q06/DuckDB:parquet | 64877528 | 5.82521e+07 | 1.11374 | ns |
| tpch_q07/DuckDB:parquet | 299170065 | 2.81431e+08 | 1.06303 | ns |
| tpch_q08/DuckDB:parquet | 308211986 | 3.23394e+08 | 0.953054 | ns |
| tpch_q09/DuckDB:parquet | 373530109 | 3.12816e+08 | 1.19409 | ns |
| tpch_q10/DuckDB:parquet | 246413231 | 2.96132e+08 | 0.832107 | ns |
| tpch_q11/DuckDB:parquet | 233163917 | 1.92984e+08 | 1.2082 | ns |
| tpch_q12/DuckDB:parquet | 109035818 | 1.04156e+08 | 1.04685 | ns |
| tpch_q13/DuckDB:parquet | 313100898 | 2.75915e+08 | 1.13477 | ns |
| tpch_q14/DuckDB:parquet | 132512578 | 1.04378e+08 | 1.26955 | ns |
| tpch_q15/DuckDB:parquet | 109381063 | 8.31596e+07 | 1.31531 | ns |
| tpch_q16/DuckDB:parquet | 131903417 | 1.31087e+08 | 1.00623 | ns |
| tpch_q17/DuckDB:parquet | 187073184 | 1.46104e+08 | 1.28041 | ns |
| tpch_q18/DuckDB:parquet | 273853570 | 2.74338e+08 | 0.998236 | ns |
| tpch_q19/DuckDB:parquet | 138483291 | 1.45102e+08 | 0.954387 | ns |
| tpch_q20/DuckDB:parquet | 235797520 | 2.12129e+08 | 1.11158 | ns |
| tpch_q21/DuckDB:parquet | 408336192 | 3.51901e+08 | 1.16037 | ns |
| tpch_q22/DuckDB:parquet | 165759321 | 1.33434e+08 | 1.24226 | ns |
Random access is quite a big regression too. I would have expected the opposite. Is that also the same thing as clickbench regressions?
There is only a regression on random access which I will now look into
Its hard to measure exact what is going on, but my guess it that row intersections are slower than a btree.
Deploying vortex-bench with 
Cloudflare Pages
| Latest commit: |
194ffc7
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://e16ba5e0.vortex-93b.pages.dev |
| Branch Preview URL: | https://ji-layout-mask-stream.vortex-93b.pages.dev |
![cloudflare-workers-and-pages[bot] avatar](https://avatars.githubusercontent.com/in/85455?v=4 "Published by a pub.dev verified publisher"){kind=small}