atlas icon indicating copy to clipboard operation
atlas copied to clipboard

Update objectPrefix isUnique for aws_s3_v2_directory to false

Open AmeerAssi opened this issue 4 years ago • 1 comments

objectPrefix attribute in aws_s3_v2_directory is a required field, in addition its isUnique property is true.

based on the test here: (see verifyS3V2PseudoDir) https://github.com/apache/atlas/blob/d27790dfdde09a58db15064bbdaf77d224f61ecc/common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java

if we have 2 AWS S3 files from different buckets:

  1. s3://aws_my_bucket1/1234567890/test.csv
  2. s3://aws_my_bucket2/1234567890/test.csv

as you see there is 2 different directories from different buckets, but having same objectPrefix, and different qualified name.

object 1: aws_s3_v2_directory: qualifiedName: "s3://aws_my_bucket1/1234567890/" objectPrefix: "/1234567890/"

object 2: aws_s3_v2_directory: qualifiedName: "s3://aws_my_bucket2/1234567890/" objectPrefix: "/1234567890/"

which making the objectPrefix isUnique property conflicting with qualified name property.

AmeerAssi avatar Mar 10 '21 05:03 AmeerAssi

This change has been merged already. @AmeerAssi Please close the PR.

sidharthkmishra avatar Mar 29 '21 22:03 sidharthkmishra

Fix merged via https://issues.apache.org/jira/browse/ATLAS-4227.

mneethiraj avatar Oct 10 '22 06:10 mneethiraj