datafusion-ballista
datafusion-ballista copied to clipboard
Add a feature based object store provider
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently ObjectStore cannot be self-registered by its path url, which is very important feature for Ballista to support remote storage, like HDFS, etc.
Describe the solution you'd like
By introduce FeatureBasedObjectStoreProvider, we can just add features for different object stores for the Ballista. And then we can detect which object store to be used by URL without do any manual object store registration.
Describe alternatives you've considered
Additional context
https://github.com/apache/arrow-datafusion/issues/2905