open-local icon indicating copy to clipboard operation
open-local copied to clipboard

kube-scheduler flag policy-config-file removed from v1.23

Open chinglinwen opened this issue 3 years ago • 2 comments

Ⅰ. Issue Description

Current oepn-local helm chart using job to append flag policy-config-file to kube-scheduler, and the flag is removed from v1.23, cause error.

Ⅱ. Describe what happened

Scheduler crash

Error: unknown flag: --policy-config-file

https://sourcegraph.com/github.com/kubernetes/kubernetes/-/blob/CHANGELOG/CHANGELOG-1.23.md?L1974

The legacy scheduler policy config is removed in v1.23, the associated flags policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (#105424, @kerthcet) [SIG Scheduling and Testing]

Ⅲ. Describe what you expected to happen

Scheduler running success .

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. helm template openlocal ./helm > openlocal.yaml
  2. kubectl apply open-local.yaml

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

  • k8s version: v1.23.6
  • Open-Local version: v0.5.4
  • OS (e.g. from /etc/os-release): centos8
  • Kernel (e.g. uname -a): Linux 4.18.0-305.3.1.el8.x86_64 SMP Tue Jun 1 16:14:33 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others:

chinglinwen avatar May 16 '22 08:05 chinglinwen

Doing this, seems ok, it seems not compatible with KubeVirt's DataVolume? https://github.com/kubevirt/kubevirt/issues/7754

kube-schedule config

[root@hcit3 ~]# cat /etc/kubernetes/scheduler-config.yaml
apiVersion: kubescheduler.config.k8s.io/v1beta2
kind: KubeSchedulerConfiguration
clientConnection:
  kubeconfig: /etc/kubernetes/scheduler.conf
extenders:
  - urlPrefix: "http://open-local-scheduler-extender.openlocal:23000/scheduler"
    filterVerb: "predicates"
    prioritizeVerb: "priorities"
    preemptVerb: ""
    bindVerb: ""
    weight: 10
    enableHTTPS: false
    nodeCacheCapable: true
    ignorable: true

[root@hcit3 ~]# cat /etc/kubernetes/manifests/kube-scheduler.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-scheduler
    tier: control-plane
  name: kube-scheduler
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-scheduler
    - --config=/etc/kubernetes/scheduler-config.yaml
    - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
    - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
    - --bind-address=127.0.0.1
    - --kubeconfig=/etc/kubernetes/scheduler.conf
    #- --policy-config-file=/etc/kubernetes/scheduler-policy-config.json
    - --leader-elect=true
    image: ecr-sh.yun.example.cn/kubevirt/kube-scheduler:v1.23.6
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10259
        scheme: HTTPS
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 15
    name: kube-scheduler
    resources:
      requests:
        cpu: 100m
    startupProbe:
      failureThreshold: 24
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10259
        scheme: HTTPS
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 15
    volumeMounts:
    - mountPath: /etc/kubernetes/scheduler-policy-config.json
      name: scheduler-policy-config
      readOnly: true
    - mountPath: /etc/kubernetes/scheduler-config.yaml
      name: scheduler-config
      readOnly: true
    - mountPath: /etc/kubernetes/scheduler.conf
      name: kubeconfig
      readOnly: true
  hostNetwork: true
  dnsPolicy: ClusterFirstWithHostNet
  priorityClassName: system-node-critical
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  volumes:
  - hostPath:
      path: /etc/kubernetes/scheduler-policy-config.json
      type: File
    name: scheduler-policy-config
  - hostPath:
      path: /etc/kubernetes/scheduler-config.yaml
      type: File
    name: scheduler-config
  - hostPath:
      path: /etc/kubernetes/scheduler.conf
      type: FileOrCreate
    name: kubeconfig
status: {}

chinglinwen avatar May 17 '22 03:05 chinglinwen

Thx for your workaround. It would be very grateful if you can fix this, we need more great contributors like you.

TheBeatles1994 avatar Jun 13 '22 03:06 TheBeatles1994