polars
polars copied to clipboard
feat(rust): streaming parquet from object_stores
Supports reading parquet files with the object_store crate. Currently the file
and s3
protocols are linked in, gcs and azure can be added if this approach makes sense.
The only supported mode is batched
since generally remote files are big. Setting a big chunk size will download all the file in one iteration.
This is my first PR in this repo, feedback appreciated.