pinot icon indicating copy to clipboard operation
pinot copied to clipboard

API to expose the count of active task executions on minion

Open shounakmk219 opened this issue 1 year ago • 1 comments

Description

Expose an API that provides the value of MinionGauge.NUMBER_OF_TASKS gauge which corresponds to the number of active task executions going on the minion.

Path : /tasks/active/count Method : GET Sample response:

{
  "activeTasksCount": 10
}

shounakmk219 avatar Jul 05 '24 13:07 shounakmk219

Codecov Report

Attention: Patch coverage is 0% with 11 lines in your changes missing coverage. Please review.

Project coverage is 61.96%. Comparing base (59551e4) to head (2586796). Report is 782 commits behind head on master.

Files Patch % Lines
...nion/api/resources/PinotMinionMetricsResource.java 0.00% 9 Missing :warning:
...g/apache/pinot/common/metrics/AbstractMetrics.java 0.00% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13550      +/-   ##
============================================
+ Coverage     61.75%   61.96%   +0.21%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2554     +118     
  Lines        133233   140482    +7249     
  Branches      20636    21852    +1216     
============================================
+ Hits          82274    87054    +4780     
- Misses        44911    46797    +1886     
- Partials       6048     6631     +583     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration <0.01% <0.00%> (-0.01%) :arrow_down:
integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration2 0.00% <0.00%> (ø)
java-11 61.91% <0.00%> (+0.20%) :arrow_up:
java-21 61.85% <0.00%> (+0.22%) :arrow_up:
skip-bytebuffers-false 61.95% <0.00%> (+0.21%) :arrow_up:
skip-bytebuffers-true 61.81% <0.00%> (+34.08%) :arrow_up:
temurin 61.96% <0.00%> (+0.21%) :arrow_up:
unittests 61.96% <0.00%> (+0.21%) :arrow_up:
unittests1 46.43% <0.00%> (-0.46%) :arrow_down:
unittests2 27.75% <0.00%> (+0.02%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jul 05 '24 13:07 codecov-commenter

That makes sense. Will make the changes accordingly.

shounakmk219 avatar Jul 11 '24 05:07 shounakmk219

@Jackie-Jiang I explored the option of having an endpoint to get metric values. There were lot of other complications around it hence decided to stick to the existing API just instead of relying on the gauge metric it fetches the active tasks count using MinionEventObservers

shounakmk219 avatar Jul 12 '24 10:07 shounakmk219

@Jackie-Jiang based on your inputs updated the code to expose just the gauge values for now.

shounakmk219 avatar Jul 16 '24 12:07 shounakmk219