Nicholas Gates

Clean up stats propagation for slicing

1

Reduces the amount we copy some stats (by removing into_iter that forces a full stats copy), and also makes duckdb databases idempotent.

[do not merge] Move is_constant to ArrayOperations

1

Fixes #3274

Perhaps omit file-level statistics from footer

2

If we introduce #2990 , then we may be able to serve file-level statistics from a layout, rather than have a special segment in the footer...

We expect these to be far more common than they currently are. Keeping timestamps as extension dtypes is part of the forcing function to ensure better usability. We'll track the...

Add LST Bench

1

https://github.com/microsoft/lst-bench

Similar to Patches + Validity, create a new Offsets helper for run-end, list offsets, etc. Being able to group-by offset, slice, etc. can be shared. We should also cache canonicalized...

When writing, we want to eagerly assign segment IDs, but some layouts want to ensure their data ends up near the footer. In these cases we allow a layout to...

Add the ability to easily repartition an array stream by row count. May produce odd arrays by slicing and combining using Chunked. While this is zero-copy. The performance of chunked...

Slice implementations should return ConstantArray where possible

2

For example, dict array. Then we can remove this awkward experiment around cost