snowplow-rdb-loader icon indicating copy to clipboard operation
snowplow-rdb-loader copied to clipboard

Published 20 hours ago • verified publisher icon snowplow

Common: explore S3Guard to improve S3 consistency

Open chuwy opened this issue 8 years ago • 0 comments

S3Guard is an experimental s3a-plugin that uses DynamoDB as an intermediate metadata store.

Right now this is part of Hadoop Common 2.9 and should be considered unstable. Also I'm not sure that our S3DistCp approach can work with s3a.

See also for current approach: https://github.com/snowplow/snowplow/issues/3113

chuwy avatar Nov 03 '17 05:11 chuwy