iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Published 20 hours ago • verified publisher icon apache

AWS: Support preload S3 client mode for S3FileIO

Open xiaoxuandev opened this issue 3 years ago • 3 comments

Adding support for Initializing S3 client during the S3FileIO initialization. This is needed to deal with a scenario in which S3 files need to be deleted after table has been dropped when LakeFormation is enabled.

xiaoxuandev avatar Aug 12 '22 00:08 xiaoxuandev

should we also include the code part which actually does s3 client initialization based on this property in this PR ?

singhpk234 avatar Aug 12 '22 06:08 singhpk234

should we also include the code part which actually does s3 client initialization based on this property in this PR ?

good point, I discussed and we want to separate that part because it requires merging the LakeFormation caching PR #4695 first. This PR is to reduce the volume of that one.

jackye1995 avatar Aug 12 '22 07:08 jackye1995

Oh actually nvm, @singhpk234 I get what you mean now, yes the part in S3FileIO should be included. @xiaoxuandev let us know when you have that part added.

jackye1995 avatar Aug 12 '22 07:08 jackye1995

Looks good to me!

ericlgoodman avatar Aug 15 '22 19:08 ericlgoodman

Thanks for all the reviews, looks like we have enough votes. @xiaoxuandev thanks for the contribution.

jackye1995 avatar Aug 15 '22 20:08 jackye1995