k3d
k3d copied to clipboard
Published
20 hours ago •
k3d-io
k3d-io
[QUESTION/HELP] Cannot enable DNS fix, as Host Gateway IP is missing
Question / Where do you need Help?
I'm trying to create a new cluster with k3d cluster create --network host --wait --no-lb but I hit the following error message:
INFO[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] Prep: Network
INFO[0000] Re-using existing network 'host' (1e719cb249ce6493d48c821b55507e3935eb3f12c74e4cfc84a91dc7a52be838)
INFO[0000] Created image volume k3d-k3s-default-images
INFO[0000] Starting new tools node...
INFO[0000] Starting node 'k3d-k3s-default-tools'
INFO[0001] Creating node 'k3d-k3s-default-server-0'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] Starting cluster 'k3s-default'
INFO[0001] Starting servers...
ERRO[0001] Failed Cluster Start: Failed to start server k3d-k3s-default-server-0: failed to enable k3d fixes: Cannot enable DNS fix, as Host Gateway IP is missing!
ERRO[0001] Failed to create cluster >>> Rolling Back
INFO[0001] Deleting cluster 'k3s-default'
INFO[0001] Deleting 1 attached volumes...
FATA[0001] Cluster creation FAILED, all changes have been rolled back!
Scope of your Question
k3d/k3s versions:
k3d version v5.6.3
k3s version v1.28.8-k3s1 (default)
Docker:
Client: Docker Engine - Community
Version: 26.1.1
API version: 1.45
Go version: go1.21.9
Git commit: 4cf5afa
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 26.1.1
API version: 1.45 (minimum version 1.24)
Go version: go1.21.9
Git commit: ac2de55
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.31
GitCommit: e377cd56a71523140ca6ae87e30244719194a521
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
docker-init:
Version: 0.19.0
GitCommit: de40ad0
I'm experiencing the same problem
➜ ~ docker version
Client:
Version: 26.1.2
API version: 1.45
Go version: go1.22.3
Git commit: 211e74b240
Built: Thu May 9 14:45:05 2024
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 26.1.2
API version: 1.45 (minimum version 1.24)
Go version: go1.22.3
Git commit: ef1912d8b6
Built: Thu May 9 14:45:05 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.7.16
GitCommit: 83031836b2cf55637d7abf847b17134c51b38e53.m
runc:
Version: 1.1.12
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
➜ ~ k3d version
k3d version v5.6.3
k3s version v1.28.8-k3s1 (default)
➜ ~ k3d cluster create --network host --verbose
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] Runtime Info:
&{Name:docker Endpoint:/var/run/docker.sock Version:26.1.2 OSType:linux OS:Arch Linux Arch:x86_64 CgroupVersion:2 CgroupDriver:systemd Filesystem:extfs InfoName:who}
DEBU[0000] Additional CLI Configuration:
cli:
api-port: ""
env: []
k3s-node-labels: []
k3sargs: []
ports: []
registries:
create: ""
runtime-labels: []
runtime-ulimits: []
volumes: []
hostaliases: []
DEBU[0000] Configuration:
agents: 0
image: docker.io/rancher/k3s:v1.28.8-k3s1
network: host
options:
k3d:
disableimagevolume: false
disableloadbalancer: false
disablerollback: false
loadbalancer:
configoverrides: []
timeout: 0s
wait: true
kubeconfig:
switchcurrentcontext: true
updatedefaultkubeconfig: true
runtime:
agentsmemory: ""
gpurequest: ""
hostpidmode: false
serversmemory: ""
registries:
config: ""
use: []
servers: 1
subnet: ""
token: ""
DEBU[0000] ========== Simple Config ==========
{TypeMeta:{Kind:Simple APIVersion:k3d.io/v1alpha5} ObjectMeta:{Name:} Servers:1 Agents:0 ExposeAPI:{Host: HostIP: HostPort:} Image:docker.io/rancher/k3s:v1.28.8-k3s1 Network:host Subnet: ClusterToken: Volumes:[] Ports:[] Options:{K3dOptions:{Wait:true Timeout:0s DisableLoadbalancer:false DisableImageVolume:false NoRollback:false NodeHookActions:[] Loadbalancer:{ConfigOverrides:[]}} K3sOptions:{ExtraArgs:[] NodeLabels:[]} KubeconfigOptions:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true} Runtime:{GPURequest: ServersMemory: AgentsMemory: HostPidMode:false Labels:[] Ulimits:[]}} Env:[] Registries:{Use:[] Create:<nil> Config:} HostAliases:[]}
==========================
DEBU[0000] ========== Merged Simple Config ==========
{TypeMeta:{Kind:Simple APIVersion:k3d.io/v1alpha5} ObjectMeta:{Name:} Servers:1 Agents:0 ExposeAPI:{Host: HostIP: HostPort:43869} Image:docker.io/rancher/k3s:v1.28.8-k3s1 Network:host Subnet: ClusterToken: Volumes:[] Ports:[] Options:{K3dOptions:{Wait:true Timeout:0s DisableLoadbalancer:false DisableImageVolume:false NoRollback:false NodeHookActions:[] Loadbalancer:{ConfigOverrides:[]}} K3sOptions:{ExtraArgs:[] NodeLabels:[]} KubeconfigOptions:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true} Runtime:{GPURequest: ServersMemory: AgentsMemory: HostPidMode:false Labels:[] Ulimits:[]}} Env:[] Registries:{Use:[] Create:<nil> Config:} HostAliases:[]}
==========================
INFO[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
DEBU[0000] Host network was chosen, changing provided/random api port to k3s:6443
DEBU[0000] Disabling the load balancer
DEBU[0000] ===== Merged Cluster Config =====
&{TypeMeta:{Kind: APIVersion:} Cluster:{Name:k3s-default Network:{Name:host ID: External:true IPAM:{IPPrefix:invalid Prefix IPsUsed:[] Managed:false} Members:[]} Token: Nodes:[0xc0005b2700] InitNode:<nil> ExternalDatastore:<nil> KubeAPI:0xc0004a41c0 ServerLoadBalancer:<nil> ImageVolume: Volumes:[]} ClusterCreateOpts:{DisableImageVolume:false WaitForServer:true Timeout:0s DisableLoadBalancer:true GPURequest: ServersMemory: AgentsMemory: NodeHooks:[] GlobalLabels:map[app:k3d] GlobalEnv:[] HostAliases:[] Registries:{Create:<nil> Use:[] Config:<nil>}} KubeconfigOpts:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true}}
===== ===== =====
INFO[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
DEBU[0000] Host network was chosen, changing provided/random api port to k3s:6443
DEBU[0000] '--kubeconfig-update-default set: enabling wait-for-server
INFO[0000] Prep: Network
DEBU[0000] Found network {Name:host ID:90a942a76d787f15f409d91a9248e9287654b6aa5734205926b465a2a08860f9 Created:2023-02-27 17:20:24.371795072 +0100 CET Scope:local Driver:host EnableIPv6:false IPAM:{Driver:default Options:map[] Config:[]} Internal:false Attachable:false Ingress:false ConfigFrom:{Network:} ConfigOnly:false Containers:map[] Options:map[] Labels:map[] Peers:[] Services:map[]}
DEBU[0000] Network host does not have an IPAM config
INFO[0000] Re-using existing network 'host' (90a942a76d787f15f409d91a9248e9287654b6aa5734205926b465a2a08860f9)
INFO[0000] Created image volume k3d-k3s-default-images
INFO[0000] Starting new tools node...
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] [autofix cgroupsv2] cgroupVersion: 2
DEBU[0000] [Docker] DockerHost: '' ()
DEBU[0000] Created container k3d-k3s-default-tools (ID: fb97b52d5a244087f7b892c75ba47abd200ee2902b115bdcaa303e2b890f9355)
DEBU[0000] Node k3d-k3s-default-tools Start Time: 2024-05-10 11:46:59.188147511 +0200 CEST m=+0.072300098
INFO[0000] Starting node 'k3d-k3s-default-tools'
DEBU[0000] Truncated 2024-05-10 09:46:59.32316429 +0000 UTC to 2024-05-10 09:46:59 +0000 UTC
INFO[0001] Creating node 'k3d-k3s-default-server-0'
DEBU[0001] Created container k3d-k3s-default-server-0 (ID: 3dbb8d42b080ad9234ac447c4fe221924c291e0e898f1d9742ef76941a1da233)
DEBU[0001] Created node 'k3d-k3s-default-server-0'
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
INFO[0001] Using the k3d-tools node to gather environment information
DEBU[0001] no netlabel present on container /k3d-k3s-default-tools
DEBU[0001] failed to get IP for container /k3d-k3s-default-tools as we couldn't find the cluster network
DEBU[0001] Deleting node k3d-k3s-default-tools ...
INFO[0001] Starting cluster 'k3s-default'
INFO[0001] Starting servers...
DEBU[0001] >>> enabling dns magic
ERRO[0001] Failed Cluster Start: Failed to start server k3d-k3s-default-server-0: failed to enable k3d fixes: Cannot enable DNS fix, as Host Gateway IP is missing!
ERRO[0001] Failed to create cluster >>> Rolling Back
INFO[0001] Deleting cluster 'k3s-default'
DEBU[0001] failed to get IP for container /k3d-k3s-default-server-0 as we couldn't find the cluster network
DEBU[0001] Cluster Details: &{Name:k3s-default Network:{Name:host ID:90a942a76d787f15f409d91a9248e9287654b6aa5734205926b465a2a08860f9 External:true IPAM:{IPPrefix:invalid Prefix IPsUsed:[] Managed:false} Members:[]} Token:iuqFScsHWtATjTHPVpaq Nodes:[0xc0005b2700] InitNode:<nil> ExternalDatastore:<nil> KubeAPI:0xc0004a41c0 ServerLoadBalancer:<nil> ImageVolume:k3d-k3s-default-images Volumes:[k3d-k3s-default-images]}
DEBU[0001] Deleting node k3d-k3s-default-server-0 ...
DEBU[0001] Skip deletion of cluster network 'host' because it's managed externally
INFO[0001] Deleting 1 attached volumes...
DEBU[0001] Deleting volume k3d-k3s-default-images...
FATA[0001] Cluster creation FAILED, all changes have been rolled back!
I have the same problem with the v5.6.3 just specifying the host network :
admusr@esp-26:~/cluster$ k3d --version
k3d version v5.6.3
k3s version v1.28.8-k3s1 (default)
admusr@esp-26:~/cluster$ k3d cluster create test-cluster --network host
INFO[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] Prep: Network
INFO[0000] Re-using existing network 'host' (f55e702cab7f61b535c1699c455cb9b4d18ef3386a97f9ebabb6393c8947eea3)
INFO[0000] Created image volume k3d-test-cluster-images
INFO[0000] Starting new tools node...
INFO[0000] Starting node 'k3d-test-cluster-tools'
INFO[0001] Creating node 'k3d-test-cluster-server-0'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] Starting cluster 'test-cluster'
INFO[0001] Starting servers...
ERRO[0001] Failed Cluster Start: Failed to start server k3d-test-cluster-server-0: failed to enable k3d fixes: Cannot enable DNS fix, as Host Gateway IP is missing!
ERRO[0001] Failed to create cluster >>> Rolling Back
INFO[0001] Deleting cluster 'test-cluster'
INFO[0001] Deleting 1 attached volumes...
FATA[0001] Cluster creation FAILED, all changes have been rolled back!
I see the same issue:
shuklalok@shuklalok:~$ k3d cluster create alok --network host
INFO[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] Prep: Network
INFO[0000] Re-using existing network 'host' (6bf7df481edf5a37fa5b176c413ff5dd2335a57b3e0601a4862d9b0dac057203)
INFO[0000] Created image volume k3d-alok-images
INFO[0000] Starting new tools node...
INFO[0000] Pulling image 'ghcr.io/k3d-io/k3d-tools:5.6.3'
INFO[0001] Creating node 'k3d-alok-server-0'
INFO[0001] Starting node 'k3d-alok-tools'
INFO[0001] Pulling image 'docker.io/rancher/k3s:v1.28.8-k3s1'
INFO[0003] Using the k3d-tools node to gather environment information
INFO[0003] Starting cluster 'alok'
INFO[0003] Starting servers...
ERRO[0003] Failed Cluster Start: Failed to start server k3d-alok-server-0: failed to enable k3d fixes: Cannot enable DNS fix, as Host Gateway IP is missing!
ERRO[0003] Failed to create cluster >>> Rolling Back
INFO[0003] Deleting cluster 'alok'
INFO[0003] Deleting 1 attached volumes...
FATA[0003] Cluster creation FAILED, all changes have been rolled back!
This is caused by the --network host which we had to add due to some other issue. Removing that solved the problem for me.
Have seen the same issue on the latest 5.7.3 version. Is there a fix or a workaround for this issue?
Thanks.
Any news on this? I need to add --network host and this is causing the issue. Removing works but then I have other issues so I need it and i've downgraded but thats not a solution.
Can be worked around in previous versions using K3D_FIX_DNS=0 but should be fixed in https://github.com/k3d-io/k3d/releases/tag/v5.7.4
Can be worked around in previous versions using
K3D_FIX_DNS=0but should be fixed in https://github.com/k3d-io/k3d/releases/tag/v5.7.4
Hi @iwilltry42, I'm running v5.7.4 and I'm still hitting this issue when starting the cluster. Probably those changes in v5.7.4 only fixed the problem at creation time, but the error message "Cannot enable DNS fix, as Host Gateway IP is missing!" still pops up when trying to start a previously created cluster which uses the host network.
Hey, I also just tried it out. Same error. With the K3D_FIX_DNS=0, it starts but the cluster has no connection afterwards (timeouts).
UPDATE: When I completely delete the cluster and create it again, it works. When I stop the cluster (that I have created with 5.7.4), the error appears again after starting.
INFO[0000] Using the k3d-tools node to gather environment information INFO[0000] Starting new tools node... INFO[0000] Starting node 'xxx-tools' INFO[0000] Starting cluster 'xxx' INFO[0000] Starting servers... FATA[0000] Failed to start server xxx-server-0: failed to enable k3d fixes: Cannot enable DNS fix, as Host Gateway IP is missing!
Then, I have to delete the cluster again and create it and then it works again.