bdt Add cloud object store support (S3 etc)

Add cloud object store support (S3 etc)

Open andygrove opened this issue 2 years ago • 5 comments

Jan 12 '23 15:01 andygrove

Maybe add support of storj network through uplink https://github.com/storj/uplink

Oct 16 '23 19:10 YuriyGavrilov

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

Feb 21 '24 01:02 trungda

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

Yes, it is. DataFusion supports reading from S3 and other object stores already, so hopefully this is not too hard to add.

Feb 21 '24 01:02 andygrove

@trungda I'm interested in this feature. Is this something you have solved?

May 04 '24 00:05 ForeverAngry

@trungda I'm interested in this feature. Is this something you have solved?

No I haven't yet. It's on my list but I don't have time to get to it in the next month. Feel free to jump into it?

May 06 '24 17:05 trungda