nebula icon indicating copy to clipboard operation
nebula copied to clipboard

job use future

Open panda-sheep opened this issue 2 years ago • 6 comments

fix comments

What type of PR is this?

  • [ ] bug
  • [ ] feature
  • [x] enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

Use future to make meta jobs of different spaces execute in parallel note: Because the balance leader is for all spaces, it will not be dealt with here. Same as https://github.com/vesoft-inc/nebula-ent/pull/1259 Also resolved issue https://github.com/vesoft-inc/nebula-ent/issues/867

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • [ ] Unit test(positive and negative cases)
  • [ ] Function test
  • [ ] Performance test
  • [ ] N/A

Affects:

  • [ ] Documentation affected (Please add the label if documentation needs to be modified.)
  • [ ] Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • [ ] If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • [ ] Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

panda-sheep avatar Sep 16 '22 09:09 panda-sheep

cannot stop a failed job?

(root@nebula) [test]> show jobs;
+--------+------------------+------------+----------------------------+----------------------------+
| Job Id | Command          | Status     | Start Time                 | Stop Time                  |
+--------+------------------+------------+----------------------------+----------------------------+
| 14     | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T11:12:54.000000 | 2022-09-19T11:13:25.000000 |
| 13     | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T11:11:49.000000 | 2022-09-19T11:12:54.000000 |
| 12     | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T09:29:16.000000 | 2022-09-19T09:29:21.000000 |
| 11     | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T09:29:21.000000 | 2022-09-19T11:11:49.000000 |
| 10     | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:15:59.000000 | 2022-09-19T08:15:59.000000 |
| 9      | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T08:15:59.000000 | 2022-09-19T09:29:16.000000 |
| 8      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:14:18.000000 | 2022-09-19T08:14:23.000000 |
| 7      | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T08:14:59.000000 | 2022-09-19T08:15:59.000000 |
| 6      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:07:32.000000 | 2022-09-19T08:07:37.000000 |
| 5      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:05:26.000000 | 2022-09-19T08:05:31.000000 |
| 4      | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T08:05:31.000000 | 2022-09-19T08:07:32.000000 |
| 3      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:04:04.000000 | 2022-09-19T08:04:09.000000 |
| 2      | "ZONE_BALANCE"   | "STOPPED"  | 2022-09-19T08:07:37.000000 | 2022-09-19T08:10:26.000000 |
+--------+------------------+------------+----------------------------+----------------------------+
Got 13 rows (time spent 2745/3430 us)

Tue, 20 Sep 2022 10:11:12 CST

(root@nebula) [test]> stop job 14;
[ERROR (-1005)]: Finished job or failed job can not be stopped, please start another job instead

Tue, 20 Sep 2022 10:11:17 CST

We will not be able to start a new balance job if a failed job cannnot be stopped.

kikimo avatar Sep 20 '22 02:09 kikimo

cannot stop a failed job?

When I tested it in the enterprise version, it was OK. Maybe a pr is not synced to the community version.

I take a look. thx

panda-sheep avatar Sep 20 '22 02:09 panda-sheep

@kikimo https://github.com/vesoft-inc/nebula-ent/pull/861. this pr is not synced to the community version.

panda-sheep avatar Sep 20 '22 02:09 panda-sheep

Codecov Report

Base: 84.67% // Head: 84.70% // Increases project coverage by +0.02% :tada:

Coverage data is based on head (11379f5) compared to base (7d5e93f). Patch coverage: 92.73% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4654      +/-   ##
==========================================
+ Coverage   84.67%   84.70%   +0.02%     
==========================================
  Files        1358     1358              
  Lines      135654   135694      +40     
==========================================
+ Hits       114868   114934      +66     
+ Misses      20786    20760      -26     
Impacted Files Coverage Δ
src/meta/processors/job/BalanceJobExecutor.h 75.00% <ø> (ø)
src/meta/processors/job/BalancePlan.h 100.00% <ø> (ø)
src/meta/processors/job/JobExecutor.h 100.00% <ø> (+50.00%) :arrow_up:
src/meta/processors/job/JobManager.h 100.00% <ø> (ø)
src/meta/processors/job/MetaJobExecutor.h 83.33% <81.81%> (-16.67%) :arrow_down:
src/meta/processors/job/JobManager.cpp 75.64% <84.88%> (+1.12%) :arrow_up:
...c/meta/processors/job/LeaderBalanceJobExecutor.cpp 86.90% <85.71%> (-0.08%) :arrow_down:
src/meta/processors/job/ZoneBalanceJobExecutor.cpp 75.00% <90.90%> (+0.22%) :arrow_up:
src/meta/processors/job/BalanceJobExecutor.cpp 69.67% <100.00%> (+3.00%) :arrow_up:
src/meta/processors/job/BalancePlan.cpp 72.53% <100.00%> (-3.13%) :arrow_down:
... and 55 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov-commenter avatar Sep 21 '22 02:09 codecov-commenter

generally LGTM

liwenhui-soul avatar Sep 21 '22 02:09 liwenhui-soul

Don't need merge first, wait for @critical27 to take a look

panda-sheep avatar Sep 21 '22 03:09 panda-sheep