filter out those nodes which are UnschedulableAndUnresolvable when preempting

[lowang-bh]

Fixes #3071 Part of enhancement for preemption action #3074 Those nodes with status UnschedulableAndUnresolvable is not helpful for preempting. So we can use cached predicating result to reduce the nodes numbers when scan for a victim pod.

we can also use the new Api NewFitStatus to return an error in predicating to fix issue Predicate function retrun value is not compatiable with old plugins #3086

main changes: commit-1: add predicate result code to task's fitError commit-2: testcase about fitErrors commit-3: filter out those nodes which are not helpful for preemption in preempt/reclaim actions

[lowang-bh]

/assign @Monokaix @hwdef @wangyang0616

[lowang-bh]

/assign @Monokaix @hwdef @wangyang0616 @william-wang

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: william-wang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~pkg/scheduler/OWNERS~~ [william-wang]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment