Is it karmada support deploy in IPv4/IPv6 dual-stack mode

[SZT0728]

is karmada:v1.2.0 support deploy in IPv4/IPv6 dual-stack mode? if not, which version support?

[RainbowMango]

Could you please tell us more details about your case? I'm afraid I can't get your point.

[SZT0728]

Could you please tell us more details about your case? I'm afraid I can't get your point.

i want to install karmada in the ipv6 kubernetes ,and join the ipv6 kubernetes cluster. does karmada support?

[RainbowMango]

To be honest, I'm not sure since I never tested it before. The only difference between IPv4 and IPv6 is the address of kube-apiserver, given we are using client-go(i bet it supports IPv6) to talk to the member cluster, so I guess it should be OK.

Can you help to test it and give us feedback? (I don't have the environment on hand.)

[SZT0728]

To be honest, I'm not sure since I never tested it before. The only difference between IPv4 and IPv6 is the address of kube-apiserver, given we are using client-go(i bet it supports IPv6) to talk to the member cluster, so I guess it should be OK.

Can you help to test it and give us feedback? (I don't have the environment on hand.)

the etcd start error with the log: unexpected error setting up listen-peer-urls: URL address does not have the form "host:port": http://fd74:ca9b:172:18:63cf:ab87:ceb8:1807:2380 how can i deal with it?

[RainbowMango]

I guess this error was raised during the installation, right? which tool are you using? Karmadactl or chart? Could you please show more logs here?

[SZT0728]

I guess this error was raised during the installation, right? which tool are you using? Karmadactl or chart? Could you please show more logs here?

yes, it was raise when i exec kubectl karmada init --crds=crdspath. i use the tool names kubectl-karmada. no more logs,just this line. I get the etcd statefulSet in yaml and the etcd exec command with args listen-peer-urls: http://${POD_IP}:2380. But the pod ip is ipv6-mode,it should identify like http://[${POD_IP}]:2380.
Is this problem because ETCD does not handle ipv6 format？

[RainbowMango]

Do you have a Kubernetes running in IPv6 mode? Can you show me what's the --listen-peer-urls look like?

We just fixed a similar issue by #2450 last week. @duanmengkk Have you ever met this issue?

[SZT0728]

Do you have a Kubernetes running in IPv6 mode? Can you show me what's the --listen-peer-urls look like?

We just fixed a similar issue by #2450 last week. @duanmengkk Have you ever met this issue?

The Kubernetes running in IPv6 mode show the etcd args - --listen-peer-urls=https://[4001:250:250:250:250:250:250:222]:2380 I recompile a kubectl-karmada according to https://github.com/karmada-io/karmada/pull/2450/commits/bd6b872a8d8965b983199ee72d3f686751412fc0 to see if I can fix it

[RainbowMango]

Good luck, looking forward to your feedback.

[duanmengkk]

Do you have a Kubernetes running in IPv6 mode? Can you show me what's the --listen-peer-urls look like?

We just fixed a similar issue by #2450 last week. @duanmengkk Have you ever met this issue?

No,I didn‘t met it.’May be we can open an issue such as Support ipv6 in karmada to track it ，rather than resolve it one by one

[SZT0728]

Good luck, looking forward to your feedback.

The karmada currently not support deploy in the dual-stack kubernetes well because of the etcd. The etcd start failed with the follow: "error":"--initial-cluster has etcd-0=http://etcd-0.etcd.karmada-system.svc.cluster.local:2380 but missing from --initial-advertise-peer-urls=http://[fd74:ca9b:172:18:ee5a:9c44:af76:29aa]:2380 (\"http://[fd74:ca9b:172:18:ee5a:9c44:af76:29aa]:2380\"(resolved from \"http://[fd74:ca9b:172:18:ee5a:9c44:af76:29aa]:2380\") != \"http://10.244.0.232:2380\"(resolved from \"http://etcd-0.etcd.karmada-system.svc.cluster.local:2380\"))

And this error cause by the etcd resolve domain and it return the first address default. https://github.com/etcd-io/etcd/blob/d42e8589e1305d893eeec9e7db746f6f4a76c250/pkg/netutil/netutil.go#L37 So in the dual-stack kubernetes it return the IPv4 rather than the IPv6.

[RainbowMango]

Thanks for your analysis. Do you know what we can do here? also cc author @prodanlabs for comment.

[duanmengkk]

The actual reason is the server of etcd is listened in ipv4 mode,see the etcd.conf kubectl logs -f etcd-0 -n karmada-system-test2 etcd-init-conf However,the ip address resolved by core-dns is ipv6, nslookup etcd-0.etcd.karmada-system-test2.svc.cluster.local

you can manually change the service of etcd to fix the problem temporarily.

  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack

[duanmengkk]

We can add a flag such as karmadactl init --mode ipv6 or dual-stack to fix it.But I'm not so sure that's a reasonable solution. @RainbowMango @prodanlabs The Helm Install also have the same problem.https://github.com/karmada-io/karmada/blob/789a44acfaa71159a50e1c650b89a5746a15f1a8/charts/karmada/templates/etcd.yaml#L91

[SZT0728]

The actual reason is the server of etcd is listened in ipv4 mode,see the etcd.conf kubectl logs -f etcd-0 -n karmada-system-test2 etcd-init-conf However,the ip address resolved by core-dns is ipv6, nslookup etcd-0.etcd.karmada-system-test2.svc.cluster.local

you can manually change the service of etcd to fix the problem temporarily.

  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack

sorry, Here's a bug I forgot to mention. I recompile the kubectl-karmada and add the symbol [] to the listen-peer-urls、listen-client-urls、listen-client-advertise-peer-urls for the IPv6 deploy. see the etcd config kubectl logs -f etcd-0 -n karmada-system etcd-init-conf

And the etcd service describe as below:

and the etcd start failed due to the etcd resolve above, it resolve http://etcd-0.etcd.karmada-system.svc.cluster.local:2380 return the first result default. And the first result is IPv4，not equal the initial-advertise-peer-urls which value in IPv6 mode

[SZT0728]

it translate the etcd-0 domain and the etcd resolve it and return the first result

https://github.com/etcd-io/etcd/blob/d42e8589e1305d893eeec9e7db746f6f4a76c250/pkg/netutil/netutil.go#L37

发自我的iPhone

------------------ Original ------------------ From: duanmengkk @.> Date: Tue,Oct 11,2022 4:37 PM To: karmada-io/karmada @.> Cc: SZT0728 @.>, Author @.> Subject: Re: [karmada-io/karmada] Is it karmada support deploy in IPv4/IPv6dual-stack mode (Issue #2569)

it resolve http://etcd-0.etcd.karmada-system.svc.cluster.local:2380 return the first result default

you mean nslookup etcd-0.etcd.karmada-system.svc.cluster.local returned a ipv4 address? In my env,it is a ipv6 address

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>