测试LoadBalancer类型Service没有生成lb-svc-test pod

[luoMonkeyKing]

Bug Report

根据v1.12.x文档测试lb svc，创建完service后没有生成lb-svc-test-service pod，文档地址：https://kubeovn.github.io/docs/v1.12.x/guide/loadbalancer-service/

Actual Behavior

Steps to Reproduce the Problem

1.根据文档已安装multus、开启了--enable-lb-svc=true开关 2.创建了NetworkAttachmentDefinition CRD 资源，内容和文档中一样，只是把网卡改成本地物理网卡名称 3.创建attach-subnet子网，内容和文档中一样，安装官方文档复制 4.创建LoadBalancer Service，内容和文档中一样，安装官方文档复制 5.创建完LoadBalancer Service之后，查看kube-ovn controller pod日志如下：

I0318 11:18:48.461690       1 subnet.go:299] subnet attach-subnet is not ovn subnet, no vpc
I0318 11:18:48.461715       1 subnet.go:332] format subnet attach-subnet, changed true
I0318 11:18:48.572170       1 event.go:298] Event(v1.ObjectReference{Kind:"Subnet", Namespace:"", Name:"attach-subnet", UID:"e0914810-1d17-4cd3-92b4-883e70a36055", APIVersion:"kubeovn.io/v1", ResourceVersion:"1327479", FieldPath:""}): type: 'Normal' reason: 'SubnetGatewayTypeChanged' subnet gateway type changes from "" to "distributed"
I0318 11:18:49.938553       1 event.go:298] Event(v1.ObjectReference{Kind:"Subnet", Namespace:"", Name:"attach-subnet", UID:"e0914810-1d17-4cd3-92b4-883e70a36055", APIVersion:"kubeovn.io/v1", ResourceVersion:"1327479", FieldPath:""}): type: 'Normal' reason: 'ValidateLogicalSwitchSuccess' 
I0318 11:18:50.239613       1 ipam.go:311] adding new subnet attach-subnet
I0318 11:18:50.281573       1 event.go:298] Event(v1.ObjectReference{Kind:"Subnet", Namespace:"", Name:"attach-subnet", UID:"e0914810-1d17-4cd3-92b4-883e70a36055", APIVersion:"kubeovn.io/v1", ResourceVersion:"1327479", FieldPath:""}): type: 'Normal' reason: 'SetNonOvnSubnetSuccess' 
I0318 11:18:50.607030       1 subnet.go:738] non ovn subnet attach-subnet is ready
I0318 11:18:50.607168       1 subnet.go:299] subnet attach-subnet is not ovn subnet, no vpc
I0318 11:18:50.607187       1 subnet.go:332] format subnet attach-subnet, changed false
I0318 11:18:50.607651       1 event.go:298] Event(v1.ObjectReference{Kind:"Subnet", Namespace:"", Name:"attach-subnet", UID:"e0914810-1d17-4cd3-92b4-883e70a36055", APIVersion:"kubeovn.io/v1", ResourceVersion:"1327491", FieldPath:""}): type: 'Normal' reason: 'ValidateLogicalSwitchSuccess' 
I0318 11:18:50.648492       1 event.go:298] Event(v1.ObjectReference{Kind:"Subnet", Namespace:"", Name:"attach-subnet", UID:"e0914810-1d17-4cd3-92b4-883e70a36055", APIVersion:"kubeovn.io/v1", ResourceVersion:"1327491", FieldPath:""}): type: 'Normal' reason: 'SetNonOvnSubnetSuccess' 
I0318 11:18:50.922867       1 subnet.go:738] non ovn subnet attach-subnet is ready
I0318 11:19:07.191245       1 endpoint.go:106] update add/update endpoint default/test-service
I0318 11:19:08.818589       1 endpoint.go:106] update add/update endpoint default/test-service
I0318 11:19:09.145866       1 service.go:318] handle update service default/test-service
I0318 11:19:09.145965       1 service.go:386] add vip 10.109.170.80:80 to LB cluster-tcp-loadbalancer
I0318 11:19:09.146048       1 endpoint.go:106] update add/update endpoint default/test-service

6、subnet信息：

AME                       PROVIDER                               VPC           PROTOCOL   CIDR             PRIVATE   NAT     DEFAULT   GATEWAYTYPE   V4USED   V4AVAILABLE   V6USED   V6AVAILABLE   EXCLUDEIPS                      U2OINTERCONNECTIONIP
attach-subnet              lb-svc-attachment.kube-system                        IPv4       172.18.0.0/16    false     false   false     distributed   0        65524         0        0             ["172.18.0.0..172.18.0.10"]     
join                       ovn                                    ovn-cluster   IPv4       100.64.0.0/16    false     false   false     distributed   2        65531         0        0             ["100.64.0.1"]                  
ovn-default                ovn                                    ovn-cluster   IPv4       10.16.0.0/16     false     true    true      distributed   5        65528         0        0             ["10.16.0.1"]                   
ovn-vpc-external-network   ovn-vpc-external-network.kube-system                 IPv4       192.168.0.0/24   false     false   false     distributed   1        243           0        0             ["192.168.0.1..192.168.0.10"]   

7、net-attach-def信息：

NAMESPACE     NAME                       AGE
kube-system   lb-svc-attachment          3m5s
kube-system   ovn-vpc-external-network   13h

8.svc信息：

NAME           TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes     ClusterIP      10.96.0.1       <none>        443/TCP        11d
test-service   LoadBalancer   10.109.170.80   <pending>     80:30900/TCP   3m52s

Additional Info

• Kubernetes version:

  Output of kubectl version:

  Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitTreeState:"clean", BuildDate:"2021-12-07T18:16:20Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
  

• kube-ovn version:

  
   v1.12.8
  

• operation-system/kernel version:

  Output of awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release: Output of uname -r:

  CentOS Linux 7 (Core)
  
  5.4.271-1.el7.elrepo.x86_64
  

[hongzhen-ma]

kube-ovn-controller 还有其他的有效 log 信息吗？可以单独过滤 service.go 文件的 log 另外，有 lb-svc 对应的 deployment 生成吗？

[luoMonkeyKing]

kube-ovn-controller 还有其他的有效 log 信息吗？可以单独过滤 service.go 文件的 log 另外，有 lb-svc 对应的 deployment 生成吗？

我打开实时日志，只输出了上面那些。没有lb-svc对应的deployment生成

[hongzhen-ma]

我在本地测试了下，没有遇到反馈的问题

kube-ovn 提供了 kind 安装的测试环境，可以执行 make kind-init; make kind-install-lb-svc 搭建环境验证。或者可以对比以上截图，排查下svc 创建后问题在哪。

[luoMonkeyKing]

我在本地测试了下，没有遇到反馈的问题

kube-ovn 提供了 kind 安装的测试环境，可以执行 make kind-init; make kind-install-lb-svc 搭建环境验证。或者可以对比以上截图，排查下svc 创建后问题在哪。

我卸载之后重新部署可以了，但是生成的lb-svc pod一直起不来，describe查看，报如下错误：

Warning  FailedCreatePodSandBox  50s (x4 over 75s)     kubelet  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ad80990db2a8ccfb6704618de4401617ea4335a8aa3580b215eb7619e52519df" network for pod "lb-svc-test-service-58688b77c-g2h7t": networkPlugin cni failed to set up pod "lb-svc-test-service-58688b77c-g2h7t_default" network: CmdAdd (shim): CNI request failed with status 400: 'could not extract the kubernetes runtime args: missing pod UID; attempted to recover it from the K8s API, but failed: pod "lb-svc-test-service-58688b77c-g2h7t" not found
': StdinData: {"capabilities":{"portMappings":true},"clusterNetwork":"/host/etc/cni/net.d/01-kube-ovn.conflist","cniVersion":"0.3.1","logLevel":"verbose","name":"multus-cni-network","runtimeConfig":{"portMappings":[]},"type":"multus-shim"}

[github-actions[bot]]

Issues go stale after 60d of inactivity. Please comment or re-open the issue if you are still interested in getting this issue fixed.

[bobz965]

我在本地测试了下，没有遇到反馈的问题 kube-ovn 提供了 kind 安装的测试环境，可以执行 make kind-init; make kind-install-lb-svc 搭建环境验证。或者可以对比以上截图，排查下svc 创建后问题在哪。

我卸载之后重新部署可以了，但是生成的lb-svc pod一直起不来，describe查看，报如下错误：

Warning  FailedCreatePodSandBox  50s (x4 over 75s)     kubelet  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ad80990db2a8ccfb6704618de4401617ea4335a8aa3580b215eb7619e52519df" network for pod "lb-svc-test-service-58688b77c-g2h7t": networkPlugin cni failed to set up pod "lb-svc-test-service-58688b77c-g2h7t_default" network: CmdAdd (shim): CNI request failed with status 400: 'could not extract the kubernetes runtime args: missing pod UID; attempted to recover it from the K8s API, but failed: pod "lb-svc-test-service-58688b77c-g2h7t" not found
': StdinData: {"capabilities":{"portMappings":true},"clusterNetwork":"/host/etc/cni/net.d/01-kube-ovn.conflist","cniVersion":"0.3.1","logLevel":"verbose","name":"multus-cni-network","runtimeConfig":{"portMappings":[]},"type":"multus-shim"}

应该是 multus 的问题，lb-svc pod 是有 e2e 覆盖的， 至少不会拉不起来 pod