DelEndpoint returned 500 Internal Server Error

[opsnull]

I installed netplugin v1.0.1 in kubernetes 1.6.2 clusters, and I found lots of error log in contivk8s.log as following:

time="2017-04-23T02:11:37+08:00" level=info msg="==> Start New Log <==
" NETNS=EMPTY
time="2017-04-23T02:11:37+08:00" level=info msg="command: DEL, cni_args: IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=nginx-ds-6ktz8;K8S_POD_INFRA_CONTAINER_ID=72a263793292c65977d8a602d20e4f3ef615eb8fade9b608e5a5fb5b89d3b27b" NETNS=EMPTY
time="2017-04-23T02:11:37+08:00" level=error msg="GET Status '500 Internal Server Error' status code 500
"
time="2017-04-23T02:11:37+08:00" level=error msg="DelEndpoint returned 500 Internal Server Error" NETNS=EMPTY

[rchirakk]

@opsnull are you able to create/delete pods ? can you send netmaster & netplugin logs ?

[rhim]

@opsnull How did you install? Currently Contiv doesn't support 1.6.2 clusters installed with kubeadm.

[yekaifeng]

Same error messages occurred to me, but the add/ del pods operations are ok. It's because k8s cni invoke del pod multiple times (2-3) with the same pod, sometimes. You can check whether the first TearDown messages is properly handled by K8S_POD_INFRA_CONTAINER_ID. Looks like a bug from k8s. I am using kubernetes v1.6.3.

================================================= time="2017-05-15T14:25:03+08:00" level=info msg="command: DEL, cni_args: IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_PO D_NAME=echo-vip-deployment-2211283314-bz9zk;K8S_POD_INFRA_CONTAINER_ID=21c4b02016ff40213be43116b7a8691e0c6471819dc10e5f3b 1d8ca47a84d17a" NETNS=100728

time="2017-05-15T14:25:06+08:00" level=info msg="==> Start New Log <==\n" NETNS=EMPTY time="2017-05-15T14:25:06+08:00" level=info msg="command: DEL, cni_args: IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_PO D_NAME=echo-vip-deployment-2211283314-bz9zk;K8S_POD_INFRA_CONTAINER_ID=21c4b02016ff40213be43116b7a8691e0c6471819dc10e5f3b 1d8ca47a84d17a" NETNS=EMPTY time="2017-05-15T14:25:06+08:00" level=error msg="GET Status '500 Internal Server Error' status code 500 \n" time="2017-05-15T14:25:06+08:00" level=error msg="DelEndpoint returned 500 Internal Server Error" NETNS=EMPTY

time="2017-05-15T14:25:36+08:00" level=info msg="==> Start New Log <==\n" NETNS=EMPTY time="2017-05-15T14:25:36+08:00" level=info msg="command: DEL, cni_args: IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=echo-vip-deployment-2211283314-bz9zk;K8S_POD_INFRA_CONTAINER_ID=21c4b02016ff40213be43116b7a8691e0c6471819dc10e5f3b1d8ca47a84d17a" NETNS=EMPTY time="2017-05-15T14:25:36+08:00" level=error msg="GET Status '500 Internal Server Error' status code 500 \n" time="2017-05-15T14:25:36+08:00" level=error msg="DelEndpoint returned 500 Internal Server Error" NETNS=EMPTY