datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Datafusion crate lacks features to enable object_store's storage options

Open DoumanAsh opened this issue 1 month ago • 1 comments

Is your feature request related to a problem or challenge?

Right now, it is not possible to propagated aws/gcp related features to object_store directly from datafusion main crate

Describe the solution you'd like

Add aws and gcp features to do so. Would require it to be added on datafusion-common and datafusion crates

Describe alternatives you've considered

Alternatively user will have to check object_store version and add this dependency manually with required features, which is possible, but would require user to always keep object_store version in sync with datafusion which is not convenient as it is not stable dependency

Additional context

Let me know if it is acceptable and I'll create PR

DoumanAsh avatar Dec 08 '25 01:12 DoumanAsh

P.s. Additional work might be needed to have both s3 and gcp registered as default stores

DoumanAsh avatar Dec 08 '25 01:12 DoumanAsh