tez
tez copied to clipboard
apache
TEZ-4508: Allow the FAIR_PARALLELISM mode to accept multiple source vertices
As explained in TEZ-4508, I expect there are valid use cases where FairShuffleVertexManager with FAIR_PARALLELISM should accept multiple SCATTER_GATHER edges.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 25m 59s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| _ master Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 17m 2s | master passed |
| +1 :green_heart: | compile | 0m 39s | master passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu122.04 |
| +1 :green_heart: | compile | 0m 32s | master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| +1 :green_heart: | javadoc | 0m 43s | master passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu122.04 |
| +1 :green_heart: | javadoc | 0m 29s | master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +0 :ok: | spotbugs | 1m 32s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 31s | master passed |
| _ Patch Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 0m 22s | the patch passed |
| +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu122.04 |
| +1 :green_heart: | javac | 0m 24s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| -0 :warning: | checkstyle | 0m 15s | tez-runtime-library: The patch generated 1 new + 12 unchanged - 0 fixed = 13 total (was 12) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu122.04 |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| +1 :green_heart: | findbugs | 0m 58s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 5m 31s | tez-runtime-library in the patch passed. |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| 58m 3s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-306/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/306 |
| JIRA Issue | TEZ-4508 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux ee80f5606113 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 5beab4ced |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05 |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-306/1/artifact/out/diff-checkstyle-tez-runtime-library.txt |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-306/1/testReport/ |
| Max. process+thread count | 1090 (vs. ulimit of 5500) |
| modules | C: tez-runtime-library U: tez-runtime-library |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-306/1/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
While testing various cases, I found this might not work when amounts of unified vertices are uneven. I will close this one once. https://github.com/apache/tez/blob/rel/release-0.10.3/tez-runtime-library/src/main/java/org/apache/tez/dag/library/vertexmanager/FairShuffleVertexManager.java#L443-L447