s3committer icon indicating copy to clipboard operation
s3committer copied to clipboard

failing test TestS3PartitionedFileListing

Open jorkzijlstra opened this issue 7 years ago • 5 comments

Assert.assertEquals("File sets should match", expectedFiles, actualFiles);

expectedFiles have the same order as the actualFiles

java.lang.AssertionError: File sets should match expected:

<[dateint=20161115/hour=13/620acc75-14f1-44a8-8cdf-c7853d816c31.parquet, 
dateint=20161115/hour=14/059f4eab-cc21-4ab5-a4e8-ceedbc9e52cf.parquet, 
dateint=20161116/hour=13/8e87ab9d-f40f-4a5f-b61e-8f613b97fd6a.parquet, 
dateint=20161116/hour=14/af18c4db-5ade-40bc-add1-c9984c22b445.parquet]> 

but was:
<[dateint=20161116/hour=14/af18c4db-5ade-40bc-add1-c9984c22b445.parquet, 
dateint=20161116/hour=13/8e87ab9d-f40f-4a5f-b61e-8f613b97fd6a.parquet, 
dateint=20161115/hour=14/059f4eab-cc21-4ab5-a4e8-ceedbc9e52cf.parquet, 
dateint=20161115/hour=13/620acc75-14f1-44a8-8cdf-c7853d816c31.parquet]>

jorkzijlstra avatar May 17 '17 15:05 jorkzijlstra

TestS3PartitionedFileListing. testTaskOutputListingWithHiddenFiles also have the same issue

jorkzijlstra avatar May 17 '17 15:05 jorkzijlstra

I see the same failures . com.netflix.bdp.s3.TestS3PartitionedFileListing > testTaskOutputListingWithHiddenFiles com.netflix.bdp.s3.TestS3PartitionedFileListing > testTaskOutputListing

What is the update on this ?

ftago avatar Jul 15 '17 03:07 ftago

@ftago I haven't heard anything from @rdblue since I created the issue

jzijlstra avatar Jul 18 '17 07:07 jzijlstra

@ftago @jorkzijlstra @jorkzijlstra Need to change the expected and actual lists to sets - so order wont matter

yiftachdy avatar Jul 20 '17 13:07 yiftachdy

I think there's an easy fix to use sets. If you open a PR, I can merge the fix.

rdblue avatar Aug 08 '18 23:08 rdblue