datafusion-objectstore-s3

datafusion-objectstore-s3 copied to clipboard

Tests failing, update the Minio version used for testing

1

Testing on master branch using the commands provided in the README fails. The issue as suggested here Both these Issues: [StackOverflow](https://stackoverflow.com/a/73047454/8176040), [GitHub Issue](https://github.com/bitnami/bitnami-docker-minio/issues/65#issuecomment-1189916805), [Minio BlogPost](https://blog.min.io/minio-versioning-metadata-deep-dive/) seems to be that Minio...

saikrishna1-bidgely

Upgrade to the latest AWS crate.

saikrishna1-bidgely

fixes #66

saikrishna1-bidgely

Investigate using S3 Select

6

It seems support was added for this based on https://github.com/awslabs/aws-sdk-rust/releases/tag/v0.0.17-alpha Look into integrating this into `S3FileSystem` or using it to create a `TableProvider`.

matthewmturner

Create python bindings

7

matthewmturner

Uses latest version of AWS SDK crates P.S: tests fail, [observed elsewhere](https://github.com/datafusion-contrib/datafusion-objectstore-s3/pull/64#issuecomment-1156000215) as well

de-sh

hey, do we have async support to read s3 files I would see support for reading local files using async ParquetRecordBatchStreamBuilder ?

sundeepks

Experiment with rust-s3 and caching

11

This merge requests provide: - update to datafusion master branch - drop `aws-sdk-rust` in favor of much simpler `rust-s3` - Tested with self-hosted S3 servers: MinIO, SeaweedFS, s3-server (https://crates.io/crates/s3-server) (no...

mateuszkj

Test reading partitioned data

1

matthewmturner

matthewmturner