vortex
vortex copied to clipboard
spiraldb
A toolkit for working with compressed array data
We're removing until DataFusion has better support, see https://github.com/apache/datafusion/issues/10918
## 🤖 New release * `vortex-alp`: 0.12.0 -> 0.13.0 (✓ API compatible changes) * `vortex-array`: 0.12.0 -> 0.13.0 (✓ API compatible changes) * `vortex-buffer`: 0.12.0 -> 0.13.0 * `vortex-datetime-dtype`: 0.12.0...
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
Hi Vortex, I am not sure this is the desired behavior. For example, if we compress a `LargeBinary` or `LargeUtf8` Arrow Array into Vortex's `ConstantArray` and then canonicalize it back,...
There is currently a specific problem: the default writer preserves the chunking of its input; however, the default reader forces a 64Ki batch size (this is configurable but defaults to...
Its validity is either `AllValid` or `Array`, but in the latter case, it's only the first element that is false, followed by `array.len() - 1` true elements
feat(perf): DictArray.slice should act differently in different regimes of slice and dictionary size
There are at least three regimes and three values of interest: the logical length of the array (which is equal to the length of the indices), the length of the...
Adds `scalars_dtype` to `ExtDType`. This PR adds `scalars_dtype` (alternative name options: `canonical_dtype`, `storage_dtype`) to `ExtDType`. This is desirable for a few reasons * Makes it possible to canonicalize an empty...
I also remove the is_strict_sorted check because it provides litle value and is rather expensive, particularly now that we need not compute all statistics to know if a VarBinArray is...