datafusion
datafusion copied to clipboard
Datafusion crate lacks features to enable object_store's storage options
Is your feature request related to a problem or challenge?
Right now, it is not possible to propagated aws/gcp related features to object_store directly from datafusion main crate
Describe the solution you'd like
Add aws and gcp features to do so.
Would require it to be added on datafusion-common and datafusion crates
Describe alternatives you've considered
Alternatively user will have to check object_store version and add this dependency manually with required features, which is possible, but would require user to always keep object_store version in sync with datafusion which is not convenient as it is not stable dependency
Additional context
Let me know if it is acceptable and I'll create PR
P.s. Additional work might be needed to have both s3 and gcp registered as default stores