datafusion-ballista icon indicating copy to clipboard operation
datafusion-ballista copied to clipboard

Add a feature based object store provider

Open yahoNanJing opened this issue 3 years ago • 0 comments

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Currently ObjectStore cannot be self-registered by its path url, which is very important feature for Ballista to support remote storage, like HDFS, etc.

Describe the solution you'd like

By introduce FeatureBasedObjectStoreProvider, we can just add features for different object stores for the Ballista. And then we can detect which object store to be used by URL without do any manual object store registration.

Describe alternatives you've considered

Additional context

https://github.com/apache/arrow-datafusion/issues/2905

yahoNanJing avatar Sep 21 '22 06:09 yahoNanJing