hadoop
hadoop copied to clipboard
Published
20 hours ago •
apache
apache
HADOOP-18482. Update ITestS3APrefetchingInputStream to skip if CSV test file unavailable
Description of PR
HADOOP-18482
This test still runs (and fails) when we set the CSV file configuration to empty. It's meant to skip.
How was this patch tested?
- Run against bucket in
eu-west-1: passing - Run against a private S3 endpoint that does not contain file, so empty space: passing (skipping the right tests)
For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [x] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
I'm seeing the test in question fail intermittently.
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.691 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream
[ERROR] testRandomReadLargeFile(org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream) Time elapsed: 2.826 s <<< FAILURE!
org.junit.ComparisonFailure: [Gauge named stream_read_blocks_in_cache with expected value 2] expected:<[2]L> but was:<[1]L>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticValue(IOStatisticAssertions.java:257)
at org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticGaugeValue(IOStatisticAssertions.java:190)
at org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream.testRandomReadLargeFile(ITestS3APrefetchingInputStream.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
I might be breaking it when running tests. It seems to run in both sequential and parallel. and fails for sequential.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 50s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 45s | trunk passed | |
| +1 :green_heart: | compile | 0m 52s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 53s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 0m 49s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 52s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 39s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 41s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 1m 24s | trunk passed | |
| +1 :green_heart: | shadedclient | 20m 58s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 38s | the patch passed | |
| +1 :green_heart: | compile | 0m 39s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 39s | the patch passed | |
| +1 :green_heart: | compile | 0m 37s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 0m 37s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 26s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 37s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 1m 12s | the patch passed | |
| +1 :green_heart: | shadedclient | 20m 44s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 47s | hadoop-aws in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 45s | The patch does not generate ASF License warnings. | |
| 97m 23s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4983/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4983 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 411ad01a563b 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cb80a9bf607720296a5f8f85ebe168b4e3b9252a |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4983/1/testReport/ |
| Max. process+thread count | 722 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4983/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
