pai icon indicating copy to clipboard operation
pai copied to clipboard

Provide a "no cni mode" in OpenPAI

Open hzy46 opened this issue 4 years ago • 3 comments

Sometimes k8s network plugin causes some problems in OpenPAI. Users can follow the steps below to solve the problem:

  1. remove CNI in Kubernetes Detailed steps are provided here : https://openpai.readthedocs.io/en/latest/manual/cluster-admin/installation-faqs-and-troubleshooting.html#how-to-remove-the-k8s-network-plugin

  2. change OpenPAI source code

    • Change pylon and rest-server to use hostNetwork
    • Change pylon/rest-server config, make them use “localhost” to connect PAI service in master node
    • Change hivedScheduler service type from “NodePort” to “ClusterIP”

The solution is not easy to perform. Maybe we can provide a "no cni mode" in OpenPAI for this need.

hzy46 avatar Apr 23 '21 02:04 hzy46

Maybe we could deploy a simple cni, such as kubenet https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#kubenet. More refer: https://docs.microsoft.com/en-us/azure/aks/configure-kubenet

I'm not sure if this could solve our issue

Binyang2014 avatar Apr 23 '21 03:04 Binyang2014

@Binyang2014 is kubenet available in Kubespray?

fanyangCS avatar Apr 23 '21 03:04 fanyangCS

kubespry doesn't support it... https://github.com/kubernetes-sigs/kubespray/blob/daed3e5b6a085ac99e076b51d314fcf76e4127b4/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml#L33

But it support a plugin called cni https://github.com/kubernetes-sigs/kubespray/blob/master/docs/cni.md. It only unpack cni binaries into /opt/cni/bin. Then we can customize kubelet to use kubenet. Seems it works from the doc

Binyang2014 avatar Apr 23 '21 04:04 Binyang2014