Calico manifest not working in the current walkthrough

[Rastusik]

Hi,

I'm trying to setup the master node according to this wakthrough: https://github.com/coreos/coreos-kubernetes/blob/master/Documentation/deploy-master.md

The problem is that I think that the Calico manifest yaml definition is wrong, because it contains a config map, a replica set and a daemon set...

I'm getting this error in my kubelet log:

Jan 05 11:00:35 coreos-kube-master-01.novalocal kubelet-wrapper[908]: E0105 10:00:35.335205     908 file_linux.go:107] can't process config file "/etc/kubernetes/manifests/calico.yaml": /etc/kubernetes/manifests/calico.yaml: read '# This ConfigMap is used to configure 

.... the yaml definiton from the guide ...

Jan 05 11:00:35 coreos-kube-master-01.novalocal kubelet-wrapper[908]: ', but couldn't parse as pod(invalid pod: &api.ConfigMap{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"calico-config", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"", ResourcceVersion:"", Generation:0, CreationTimestamp:unversioned.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*unversioned.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]api.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:""}, Data:map[string]string{"cni_network_conffig":"{\n    \"name\": \"calico\",\n    \"type\": \"flannel\",\n    \"delegate\": {\n      \"type\": \"calico\",\n      \"etcd_endpoints\": \"__ETCD_ENDPOINTS__\",\n      \"log_level\": \"info\",\n     \"policy\": {\n          \"type\": \"k8s\",\n          \"k8s_api_root\": \"https://__KUBERNETES_SERVICE_HOST__:__KUBERNETES_SERVICE_PORT__\",\n          \"k8s_auth_token\": \"__SERVICEACCOUNT_TOKEN__\"\n      },\n      \"kubernetes\": {\n          \"kubeconfig\": \"/etc/kubernetes/cni/net.d/__KUBECONFIG_FILENAME__\"\n      }\n    }\n}", "etcd_endpoints":"http://10.254.0.11:2379,http://10.254.0.12:2379"}}).

I'm running CoreOs 1185.5.0 and Kubernetes 1.5.1

Am I doing something wrong or is there an error in the guide? It seems to me that kubelet manifests should only be defined as pods, aren't they?

Thanks

[caseydavenport]

@Rastusik yeah, this looks like a bug in the documentation. The kubelet will only handle Pods from that directory, not ConfigMaps, etc.

I think the fix should be to kubectl apply that file rather than putting it in the kubelet config directory. Looks like there is a PR open already to address that here: https://github.com/coreos/coreos-kubernetes/pull/799

[ghost]

I also ran into this and the fix is indeed to not copy these files to /etc/kubernetes but the kubectl create -f ... them.

[heschlie]

@Rastusik I'll get the docs updated this week, the PR @caseydavenport referenced closed, but as stated the manifest should not live in the /etc/kubernetes/manifests/ directory and needs to be deployed with kubectl apply -f calico.yaml

Thanks for catching this.

[stephanlindauer]

i also ran into this issue. maybe it's also worth noting the difference between the manifest folder and kubectl apply, because that still isn't very clear to me.

[heschlie]

@stephanlindauer The /etc/kubernetes/manifests/ path is for static pods see here for more info. The short of it is Kubernetes will periodically scan that directory and deploy or update pods based on the manifests in there, which seems great for something like Calico, but it does not support deploying anything but pods, and we have a ConfigMap in the Calico manifest, so it fails to deploy anything in it.

[stephanlindauer]

@heschlie ah ok. understood. thanks for clearing this up. my use case is deploying a pod that is privately hosted on docker hub and therefore needs a secret to be created before. this forces me to curl everything as json to localhost on the master node once port 8080 is available. :/

[dev-e]

The docs are not updated yet.