tez
tez copied to clipboard
apache
Improve test reliability by resolving nondeterministic of HashMap
Problem
I detected the unstable For the test :
[INFO] org.apache.tez.dag.api.TestDAG#testRecreateDAG [INFO] org.apache.tez.dag.api.TestDAGPlan#userVertexOrderingIsMaintained
(they both use verify and createDag function and will be affected by the order of the vertex map)
[INFO] org.apache.tez.dag.app.dag.impl.TestVertexImpl#testVertexInit
All these tests assume the order of the Map (but java provided one with ordered specification called LinkedHashMap). which was not true, and will trigger hidden bug when the test environment changes.
Solution
we used the order-specified one which is LinkedHashMap because the order does matter for the tests (and potential future tests especially because the serialization)
Do no worry too much about the performance because these tests was detected mainly because they will be used by iteration (and the order is un defined), the iteration have better performance for LinkedHashMap: https://stackoverflow.com/questions/12998568/hashmap-vs-linkedhashmap-performance-in-iteration-over-values
Reproduce
This error can be reproduced with the NondexTool with the following command (you can try that without modifying the pom, but feel free to use the plugin to protect your project and make it safer 😊 ), and this kind problem is widely documented in International Dataset of Flaky Tests (IDoFT)
mvn edu.illinois:nondex-maven-plugin:2.1.7-SNAPSHOT:nondex -pl ./wicket-core-tests -DnondexRuns=10 -Dtest=org.apache.wicket.markup.html.form.CollectionFormComponentTest#getSetUnmodifiableSet
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 24m 52s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 6m 27s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 13m 28s | master passed |
| +1 :green_heart: | compile | 1m 19s | master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 |
| +1 :green_heart: | compile | 1m 10s | master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 25s | master passed |
| +1 :green_heart: | javadoc | 1m 13s | master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 |
| +1 :green_heart: | javadoc | 0m 57s | master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +0 :ok: | spotbugs | 1m 19s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 3s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 48s | the patch passed |
| +1 :green_heart: | compile | 0m 54s | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 |
| +1 :green_heart: | javac | 0m 54s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | javac | 0m 46s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 34s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 34s | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | findbugs | 2m 15s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 2m 13s | tez-api in the patch passed. |
| +1 :green_heart: | unit | 4m 55s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. |
| 69m 17s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-316/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/316 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux 17c0b8fccbce 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 51d6f5396 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-316/1/testReport/ |
| Max. process+thread count | 461 (vs. ulimit of 5500) |
| modules | C: tez-api tez-dag U: . |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-316/1/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
