Avoid unnecessary requeue operations in coscheduling

[Huang-Wei]

What type of PR is this?

/kind bug

What this PR does / why we need it:

The current coscheduling code aggressively actives a pod's PodGroup siblings in Permit(). For example, if a deployment doesn't reach the minMember, all its pods are in pending state. Once it's scaled up to reach the minMember, it'd trigger ActivateSiblings() N times - this causes a ton of unnecessary re-queue operations which cause the CPU usage to spike, and may relates with potential starvation reported in #682.

Which issue(s) this PR fixes:

Fixes #682

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Performance fix to eliminate unnecessary re-queue actions in coscheduling plugin

[netlify[bot]]

✅ Deploy Preview for kubernetes-sigs-scheduler-plugins canceled.

Name	Link
🔨 Latest commit	e35ce33886b1f79545cf2698906a3e4562471e20
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-scheduler-plugins/deploys/65ba8326cfb907000898b2af

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Huang-Wei

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [Huang-Wei]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[Huang-Wei]

cc @denkensk

[zwpaper]

it seems to be a valuable issue and fix to me👍

/lgtm

/hold for @denkensk

[Huang-Wei]

/cherrypick release-1.28

[k8s-infra-cherrypick-robot]

@Huang-Wei: once the present PR merges, I will cherry-pick it on top of release-1.28 in a new PR and assign it to you.

In response to this:

/cherrypick release-1.28

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-infra-cherrypick-robot]

@Huang-Wei: new pull request created: #718

In response to this:

/cherrypick release-1.28

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.