Kubeflow-operator unable to create kubeflow

[sunsingerus]

Trying to install Kubeflow with operator as described in docs, however met the following issue: operator crashes with the following log:

time="2021-03-01T13:28:59Z" level=info msg="Go Version: go1.15.1"                                                                                                               
time="2021-03-01T13:28:59Z" level=info msg="Go OS/Arch: linux/amd64"                                                                                                            
time="2021-03-01T13:28:59Z" level=info msg="Version of operator-sdk: v0.13.0"                                                                                                   
time="2021-03-01T13:28:59Z" level=info msg="Kubeflow version: 1.1.0"                                                                                                            
time="2021-03-01T13:29:01Z" level=info msg="Registering Components."                                                                                                            
time="2021-03-01T13:29:01Z" level=info msg="Adding controller for kfdef."                                                                                                       
time="2021-03-01T13:29:04Z" level=error msg="Cannot create watch for resources Deployment extensions/v1beta1: no matches for kind \"Deployment\" in version \"extensions/v1beta 
time="2021-03-01T13:29:04Z" level=info msg="Controller added to watch on Kubeflow resources with known GVK."                                                                    
time="2021-03-01T13:29:07Z" level=error msg="Could not create ServiceMonitor object. Error: servicemonitors.monitoring.coreos.com \"kubeflow-operator-957cb44b7-vx9xd-metrics\" 
time="2021-03-01T13:29:07Z" level=info msg="Starting the Cmd."                                                                                                                  
time="2021-03-01T13:29:07Z" level=info msg="Got create event for kubeflow.kubeflow."                                                                                            
time="2021-03-01T13:29:07Z" level=info msg="Watch a change for KfDef CR: kubeflow.kubeflow."                                                                                    
time="2021-03-01T13:29:08Z" level=info msg="Reconciling KfDef resources. Request.Namespace: kubeflow, Request.Name: kubeflow."                                                  
time="2021-03-01T13:29:08Z" level=info msg="Creating a new KubeFlow Deployment. KubeFlow.Namespace: kubeflow."                                                                  
time="2021-03-01T13:29:08Z" level=fatal msg="Could not get platform aws: Failed to create config file from /.kube/config **"                                                    

Manifest used kfctl_aws.v1.2.0.yaml

kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:30:33Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.15-eks-ad4801", GitCommit:"ad4801fd44fe0f125c8d13f1b1d4827e8884476d", GitTreeState:"clean", BuildDate:"2020-10-20T23:27:12Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

[moficodes]

seems like a aws config missing. aws kfdef uses some plugins.

[sunsingerus]

seems like a aws config missing.

Yes, it does. Operator writes in its log

Could not get platform aws: Failed to create config file from /.kube/config **

But there is no any instructions on how to populate this config inside operator's pod or whatever (or I can not find where to look for them).

aws kfdef uses some plugins.

Can you, please, elaborate on this?

[moficodes]

dont have a aws account / cluster to test this.

but looking at the code, in aws the get kubeconfig function expects to find a KUBECONFIG flag or the .kube/config file to be present. Which is valid when using kfctl from using local machine. But does not work inside the operator.

[danizavtz]

@sunsingerus You can populate your .kube/config folder using aws-cli, after your cluster creation on aws.

the command is:

aws eks update-kubeconfig --region <region> --name <cluster-name>

output will be something like: # Added new context arn:aws:eks:region:111122223333:cluster/my-awesome-cluster to /home/user/.kube/config

After running this command it will populate your .kube/config folder with your current kubernetes cluster configuration. And you will be able to proceed with your installation.

Ref for aws eks update-kubeconfig command documentation

[chelomontilla]

I'm having the same issue. Is there any progrees on this ?