Is there a plan to support the karmada scheduler extension

[prodanlabs]

Please provide an in-depth description of the question you have:

Recently we have a requirement that workloads are scheduled according to real-time custom metrics, such as: GPU metrics according to Prometheus.

What do you think about this question?:

I don't know much about the karmada scheduler, it seems that it is more difficult to scale, it would be great if it can be scaled as easily as the k8s scheduling framework.

[RainbowMango]

For a quick answer: Yes, there is a plan to support the scheduler extension.

We want to know more details about your use case and what's your detailed expectation. Should we meet at the community meeting and have a chat there.

cc @huone1 who is working on scheduler these days.

[huone1]

is the karmada scheduler extension dynamically extend scheduling capabilities via an http request or modify the source code of karmada-scheduler

[prodanlabs]

is the karmada scheduler extension dynamically extend scheduling capabilities via an http request or modify the source code of karmada-scheduler

For example, the workload is distributed to member1 and member2.At this time, karmada does not know the GPU usage of member1 and member2, and can only be allocated according to the expected weight.

If karmada can call a user-defined plugin, then it knows that member1 has a higher GPU usage than member2, and can automatically adjust the weights to distribute more workloads to member2

Phased extension similar to k8s scheduling framework

[lonelyCZ]

There are two ways to extend scheduling capabilities in k8s:

1. scheduler-extender: it extend scheduling capabilities via an http request, the advantage is needing not to recompile kube-scheduler.
2. scheduler framework: there are more extension point, but it need to recompile kube-scheduler.

[huone1]

is the karmada scheduler extension dynamically extend scheduling capabilities via an http request or modify the source code of karmada-scheduler

For example, the workload is distributed to member1 and member2.At this time, karmada does not know the GPU usage of member1 and member2, and can only be allocated according to the expected weight.

If karmada can call a user-defined plugin, then it knows that member1 has a higher GPU usage than member2, and can automatically adjust the weights to distribute more workloads to member2

Phased extension similar to k8s scheduling framework

I got it， you want to add a new dynamic replicas scheduling strategy to assign replicas base GPU resource usage

[prodanlabs]

2. scheduler framework:

scheduler-extender seems to be deprecated, and now most custom plugins are created based on the scheduler framework. https://github.com/kubernetes-sigs/scheduler-plugins

[huone1]

@prodanlabs let's discuss it by slack and clarify your needs. my ID is wanxing hu

[kerthcet]

Refer to kube-scheduler, two ways are quite clearly:

1. support KarmadaSchedulerConfiguration as KubeSchedulerConfiguration in kubernetes, I think this is what we want finally.
2. support out-of-tree plugins but need to compile yourself, I used to implement this feature in https://github.com/karmada-io/karmada/issues/1371

[prodanlabs]

Refer to kube-scheduler, two ways are quite clearly:

1. support KarmadaSchedulerConfiguration as KubeSchedulerConfiguration in kubernetes, I think this is what we want finally.
2. support out-of-tree plugins but need to compile yourself, I used to implement this feature in Extension for out-of-tree plugins #1371

cool, i'm looking forward to it !

[prodanlabs]

@prodanlabs let's discuss it by slack and clarify your needs. my ID is wanxing hu

Next week, I'm resting recently.

[RainbowMango]

Hi guys, let's do it at release-1.3. I added it to the backlog.

@prodanlabs please let us know how you want the karmada-scheduler to be extended.

[prodanlabs]

Hi guys, let's do it at release-1.3. I added it to the backlog.

@prodanlabs please let us know how you want the karmada-scheduler to be extended.

I'm thinking, maybe I can explain it better with a demo.

[LavredisG]

Is there a way to implement such a use case as a developer other than trying to go into the source code and create your own DynamicWeight factor (like AvailableReplicas)