atlas
atlas copied to clipboard
Update objectPrefix isUnique for aws_s3_v2_directory to false
objectPrefix attribute in aws_s3_v2_directory is a required field, in addition its isUnique property is true.
based on the test here: (see verifyS3V2PseudoDir) https://github.com/apache/atlas/blob/d27790dfdde09a58db15064bbdaf77d224f61ecc/common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java
if we have 2 AWS S3 files from different buckets:
- s3://aws_my_bucket1/1234567890/test.csv
- s3://aws_my_bucket2/1234567890/test.csv
as you see there is 2 different directories from different buckets, but having same objectPrefix, and different qualified name.
object 1: aws_s3_v2_directory: qualifiedName: "s3://aws_my_bucket1/1234567890/" objectPrefix: "/1234567890/"
object 2: aws_s3_v2_directory: qualifiedName: "s3://aws_my_bucket2/1234567890/" objectPrefix: "/1234567890/"
which making the objectPrefix isUnique property conflicting with qualified name property.
This change has been merged already. @AmeerAssi Please close the PR.
Fix merged via https://issues.apache.org/jira/browse/ATLAS-4227.