add gangPlugin and gangCache for gang scheduling

[Wenshiqi222]

Ⅰ. Describe what this PR does

1. add gangPlugin to do the scheduling at some extension points
2. add gangCache to stroe the gang info

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

• [ ] I have written necessary docs and comments
• [ ] I have added necessary unit tests and integration tests
• [ ] All checks passed in make test

[koordinator-bot[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign hormes after the PR has been reviewed. You can assign the PR to them by writing /assign @hormes in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[hormes]

@Wenshiqi222 use git commit -s to fix DCO check

[Wenshiqi222]

@Wenshiqi222 use git commit -s to fix DCO check

ok

[codecov[bot]]

Codecov Report

Merging #402 (b807fad) into main (8e49802) will increase coverage by 0.82%. The diff coverage is 83.88%.

:exclamation: Current head b807fad differs from pull request most recent head b920b40. Consider uploading reports for the commit b920b40 to get more accurate results

@@            Coverage Diff             @@
##             main     #402      +/-   ##
==========================================
+ Coverage   68.03%   68.85%   +0.82%     
==========================================
  Files         152      156       +4     
  Lines       16435    17283     +848     
==========================================
+ Hits        11181    11900     +719     
- Misses       4419     4515      +96     
- Partials      835      868      +33     

Flag	Coverage Δ
unittests	68.85% <83.88%> (+0.82%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/scheduler/plugins/gang/gang_helper.go	73.21% <73.21%> (ø)
pkg/scheduler/plugins/gang/cache.go	77.85% <77.85%> (ø)
pkg/scheduler/plugins/gang/plugin.go	82.92% <82.92%> (ø)
pkg/scheduler/plugins/gang/gang.go	90.16% <90.16%> (ø)
pkg/runtimeproxy/server/docker/utils.go	86.00% <0.00%> (-1.12%)	:arrow_down:
pkg/runtimeproxy/server/docker/handler.go	38.03% <0.00%> (+0.79%)	:arrow_up:
pkg/util/httputil/reverseproxy.go	85.10% <0.00%> (+0.79%)	:arrow_up:
pkg/runtimeproxy/server/docker/server.go	67.46% <0.00%> (+1.56%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[eahydra]

After reading the entire PR, there are a few questions that confuse me.

1. Why not extend it based on coscheduling in scheduler-plugins? The difference from coscheduling is that we need to build Gang according to Annotation, and we need to support GangGroup, and the others are no different. And the current implementation does have a lot in common with coscheduling.
2. Compared with DeniedPodGroup in coscheduling, the concept of ScheduleCycle is more complicated to understand.

I think we should consider from a longer-term development perspective, we must maintain compatibility with the community PodGroup, but also need to provide some expansion capabilities that we think are needed, although @Wenshiqi222 has done a lot of hard work (expressed here My personal sincere respects), but we should considering coscheduling based implementations.

@buptcozy @hormes @yihuifeng

[buptcozy]

After reading the entire PR, there are a few questions that confuse me.

1. Why not extend it based on coscheduling in scheduler-plugins? The difference from coscheduling is that we need to build Gang according to Annotation, and we need to support GangGroup, and the others are no different. And the current implementation does have a lot in common with coscheduling.
2. Compared with DeniedPodGroup in coscheduling, the concept of ScheduleCycle is more complicated to understand.

I think we should consider from a longer-term development perspective, we must maintain compatibility with the community PodGroup, but also need to provide some expansion capabilities that we think are needed, although @Wenshiqi222 has done a lot of hard work (expressed here My personal sincere respects), but we should considering coscheduling based implementations.

@buptcozy @hormes @yihuifeng

Besides "annotation", We offer a new "non-strict" mode to help large job get resources more easily. Also, 'Scheduling cycle' is used to solve coscheduling's incorrect calcucation in "strict mode". I think these are neccesory to imporve considering.

[eahydra]

We can copy the coscheduling into koordinator, and add these enhancements including nonStrictMode.

[Wenshiqi222]

We can copy the coscheduling into koordinator, and add these enhancements including nonStrictMode.

Ok，I will copy the Coscheduling Plugin and it's Controller code of Scheduler-Plugins to my implement and enhance it according our previous Gang's Proposal. So next week, I will rewrite all the code and adopt all the comments you @eahydra @buptcozy @yihuifeng @jasonliu747 mentioned before, really thanks for your help.

[Wenshiqi222]

Should i close this Pr and open another one? @eahydra

[jasonliu747]

Should i close this Pr and open another one? @eahydra

我认为这个 PR 可以暂时不关，然后同时提交新 PR 做对比

[jasonliu747]

/close

[koordinator-bot[bot]]

@jasonliu747: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.