parquet_s3_fdw icon indicating copy to clipboard operation
parquet_s3_fdw copied to clipboard

Published 20 hours ago • verified publisher icon pgspider

FYI: How about using arrow::fs::S3FlieSytem?

Open kou opened this issue 3 years ago • 1 comments

I see this project via pgsql-announce@.

I'm a developer of Apache Arrow. Recent Apache Arrow C++ has a integration module with S3: https://github.com/apache/arrow/blob/master/cpp/src/arrow/filesystem/s3fs.h

How about using it instead of implementing S3RandomAccessFile? We can create arrow::io::RandomAccessFile by arrow::fs::S3FileSytem::OpenInputFile().

kou avatar Mar 03 '21 22:03 kou