pinot icon indicating copy to clipboard operation
pinot copied to clipboard

return the current rebalance result if already done

Open jadami10 opened this issue 1 year ago • 1 comments
trafficstars

This is a bugfix on top of #13281 to better harden returning the rebalance result.

#13281 will likely fix the same issue, but doesn't prevent the fact that we can just forget to persist/update the job id in ZK. Since we're already waiting for the job id to persist, this PR attempts to return the actual rebalance result if it's already complete. Since Pinot 1.1 still has bugs with job id not being tracked correctly, we'll be using this internally to limit the number of times we see polling job ids fail.

jadami10 avatar Jun 26 '24 19:06 jadami10

Codecov Report

Attention: Patch coverage is 0% with 16 lines in your changes missing coverage. Please review.

Project coverage is 62.10%. Comparing base (59551e4) to head (d1e4cc3). Report is 717 commits behind head on master.

Files Patch % Lines
...oller/api/resources/PinotTableRestletResource.java 0.00% 16 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13488      +/-   ##
============================================
+ Coverage     61.75%   62.10%   +0.35%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2557     +121     
  Lines        133233   140894    +7661     
  Branches      20636    21857    +1221     
============================================
+ Hits          82274    87506    +5232     
- Misses        44911    46762    +1851     
- Partials       6048     6626     +578     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration <0.01% <0.00%> (-0.01%) :arrow_down:
integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration2 0.00% <0.00%> (ø)
java-11 62.03% <0.00%> (+0.32%) :arrow_up:
java-21 61.99% <0.00%> (+0.36%) :arrow_up:
skip-bytebuffers-false 62.06% <0.00%> (+0.31%) :arrow_up:
skip-bytebuffers-true 61.96% <0.00%> (+34.23%) :arrow_up:
temurin 62.10% <0.00%> (+0.35%) :arrow_up:
unittests 62.10% <0.00%> (+0.35%) :arrow_up:
unittests1 46.65% <ø> (-0.24%) :arrow_down:
unittests2 27.68% <0.00%> (-0.05%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jun 26 '24 19:06 codecov-commenter

In waitForJobIdToPersist(), we can consider passing in the future and stop waiting if the future is done. This way we can avoid unnecessary wait because job id will never show up

good point. added that check as well

jadami10 avatar Jul 07 '24 20:07 jadami10