hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks

Open hotcodemacha opened this issue 3 years ago • 2 comments
trafficstars

Description of PR

NumAllSinks metrics shows lower value than NumActiveSinks

JIRA - HADOOP-9946

How was this patch tested?

UT

For code changes:

  • [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hotcodemacha avatar Oct 11 '22 13:10 hotcodemacha

Thanks @ZanderXu for review and approval. @aajisaka is the original reporter for this issue, may be he can help with additional review/suggestions.

hotcodemacha avatar Oct 11 '22 14:10 hotcodemacha

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 11s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 42m 41s trunk passed
+1 :green_heart: compile 26m 0s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 27m 36s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 1m 27s trunk passed
+1 :green_heart: mvnsite 1m 55s trunk passed
+1 :green_heart: javadoc 1m 27s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 58s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 3m 0s trunk passed
+1 :green_heart: shadedclient 26m 21s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 1m 6s the patch passed
+1 :green_heart: compile 24m 54s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 24m 54s the patch passed
+1 :green_heart: compile 22m 26s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 22m 26s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 1m 19s the patch passed
+1 :green_heart: mvnsite 1m 53s the patch passed
+1 :green_heart: javadoc 1m 18s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 58s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 2m 59s the patch passed
+1 :green_heart: shadedclient 26m 1s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 18m 39s hadoop-common in the patch passed.
+1 :green_heart: asflicense 1m 10s The patch does not generate ASF License warnings.
235m 53s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/5002
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a7442b5eaeae 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 37a302496af9e96f2630206d05f7ff94fa8be561
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/testReport/
Max. process+thread count 3144 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5002/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Oct 11 '22 17:10 hadoop-yetus

@ZanderXu - Can we commit/merge this? (We dont have any other comments/suggestions)

Thanks

hotcodemacha avatar Oct 20 '22 12:10 hotcodemacha

Merged. Thanks @ashutoshcipher for your contribution and thanks @aajisaka for your review.

ZanderXu avatar Oct 25 '22 09:10 ZanderXu

But I have a little different through. Maybe the Map<String, MetricsSink> allSinks is only used to store the registered MetricsSink and sinks is used to store all the MetricsSinkAdapter.

Good point. I suppose we can simplify allSinks because it's only used to avoid duplicate sink names. Set<String> maybe sufficient instead of Map<String, MetricsSink>.

    if (allSinks.containsKey(name)) {
      LOG.warn("Sink "+ name +" already exists!");
      return sink;
    }

aajisaka avatar Oct 25 '22 12:10 aajisaka