k3d icon indicating copy to clipboard operation
k3d copied to clipboard

Published 20 hours ago • verified publisher icon k3d-io

[BUG] k3d hangs while adding node to remote cluster

Open JohnCalin opened this issue 2 years ago • 8 comments

What did you do

  • How was the cluster created?

    • On machine 1: k3d cluster create mycluster --api-port x.x.x.x:yyyy Where x.x.x.x:yyyy is the proper IP address and port I want to listen on.

  • What did you do afterwards?

    • Copied kubeconfig to machine 2 and confirmed kubectl get nodes works
    • On machine 1, fetched token with k3d cluster list --token
    • On machine 2: k3d node create myagent --role agent --cluster https://x.x.x.x:yyyy --token zzzzzzzzzzz

What did you expect to happen

A new agent node would be created on machine 2, joined to mycluster on machine 1

Screenshots or terminal output

INFO[0000] Adding 1 node(s) to the remote cluster 'https://x.x.x.x:yyyy'...
INFO[0001] Starting Node 'k3d-myagent-0'

k3d did not complete or exit. Left it for several hours without change.

Which OS & Architecture

Machine 1: CentOS Machine 2: Windows with Docker running on WSL

Which version of k3d

  • output of k3d --version On both machines:
k3d version v5.0.1-rc.1
k3s version v1.21.5-k3s2 (default)

Which version of docker

  • output of docker version and docker info Machine 1:
$ docker version
Client: Docker Engine - Community
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.8
 Git commit:        c2ea9bc
 Built:             Mon Oct  4 16:08:14 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.8
  Git commit:       79ea9d3
  Built:            Mon Oct  4 16:06:37 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 11
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.42.2.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.51GiB
 Name: XXXXXXXX
 ID: IWAK:X5YS:NSUD:R676:IPX6:BZKJ:X4HF:5HIM:EPPN:GTUE:D5CG:Y7UW
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Machine 2:

H:\>docker version
Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:58:50 2021
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:31 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

H:\>docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.0.0)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.16.3-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 9.329GiB
 Name: docker-desktop
 ID: 4W5S:OTDS:E3IP:JRZN:4TFZ:FNCF:J2EF:SU3W:LC7A:OLXD:6N3D:ZVEM
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

JohnCalin avatar Oct 13 '21 00:10 JohnCalin

Hi @JohnCalin , thanks for opening this issue! Sorry, I just found the time to go through the backlog today... Can you please paste the logs with --trace set in the k3d command, so we can debug where it's hanging? Also, please paste the output of docker logs k3d-myagent-0, as I assume it's not starting and that's why k3d is not returning (waiting for the "up and running" or "registered" log line).

There is not much testing around adding nodes to a remote cluster, so it's still considered an experimental feature / use-case.

iwilltry42 avatar Nov 15 '21 12:11 iwilltry42 

k3d version v5.4.9
k3s version v1.25.7-k3s1 (default)

The same issue, agent hangs

Logs from remote agent:

ime="2023-03-28T12:17:09Z" level=info msg="Starting k3s agent v1.25.7+k3s1 (f7c20e23)"
time="2023-03-28T12:17:09Z" level=info msg="Running load balancer k3s-agent-load-balancer 127.0.0.1:6444 -> [10.0.20.111:6500]"
time="2023-03-28T12:17:09Z" level=warning msg="Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token from the server's node-token file to enable Cluster CA validation."
The connection to the server localhost:8080 was refused - did you specify the right host or port?
time="2023-03-28T12:17:10Z" level=info msg="Module overlay was already loaded"
time="2023-03-28T12:17:10Z" level=info msg="Module nf_conntrack was already loaded"
time="2023-03-28T12:17:10Z" level=info msg="Module br_netfilter was already loaded"
time="2023-03-28T12:17:10Z" level=info msg="Module iptable_nat was already loaded"
time="2023-03-28T12:17:10Z" level=info msg="Module iptable_filter was already loaded"
time="2023-03-28T12:17:10Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400"
time="2023-03-28T12:17:10Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600"
time="2023-03-28T12:17:10Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
time="2023-03-28T12:17:10Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
time="2023-03-28T12:17:11Z" level=info msg="containerd is now running"
time="2023-03-28T12:17:11Z" level=info msg="Getting list of apiserver endpoints from server"
time="2023-03-28T12:17:11Z" level=info msg="Updating load balancer k3s-agent-load-balancer default server address -> 172.23.0.2:6443"
time="2023-03-28T12:17:11Z" level=info msg="Updating load balancer k3s-agent-load-balancer server addresses -> [172.23.0.2:6443]"
time="2023-03-28T12:17:11Z" level=info msg="Connecting to proxy" url="wss://172.23.0.2:6443/v1-k3s/connect"
time="2023-03-28T12:17:11Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=k3d-bb2-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --kubelet-cgroups=/k3s --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/etc/resolv.conf --runtime-cgroups=/k3s --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The connection to the server localhost:8080 was refused - did you specify the right host or port?
```

habibutsu avatar Mar 28 '23 12:03 habibutsu

@habibutsu I have the same issue on this version

k3d --version
k3d version v5.4.9
k3s version v1.25.7-k3s1 (default)

Running MacOS 13.1 M1 Pro.

raminqaf avatar Apr 04 '23 14:04 raminqaf

Can confirm this is still an issue even if you specify the number of agents at create time.

running in wsl2 ubuntu 22.04 on windows 11 with rancher desktop running on windows.

image

image above shows the cluster says servers and agents are ready but k3d is just hanging there.

log output

❯ /root/.k1/kubefirst/tools/k3d cluster create kubefirst --image rancher/k3s:v1.26.3-k3s1 --agents 1 --registry-create k3d-kubefirst-registry --volume /root/.k1/kubefirst/minio-storage:/var/lib/rancher/k3s/storage@all --port 443:443@loadbalancer
INFO[0000] portmapping '443:443' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy]
INFO[0000] Prep: Network
INFO[0000] Created network 'k3d-kubefirst'
INFO[0000] Created image volume k3d-kubefirst-images
INFO[0000] Creating node 'k3d-kubefirst-registry'
INFO[0000] Successfully created registry 'k3d-kubefirst-registry'
INFO[0000] Starting new tools node...
INFO[0000] Starting Node 'k3d-kubefirst-tools'
INFO[0001] Creating node 'k3d-kubefirst-server-0'
INFO[0001] Creating node 'k3d-kubefirst-agent-0'
INFO[0001] Creating LoadBalancer 'k3d-kubefirst-serverlb'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] HostIP: using network gateway 172.30.0.1 address
INFO[0001] Starting cluster 'kubefirst'
INFO[0001] Starting servers...
INFO[0001] Starting Node 'k3d-kubefirst-server-0'
INFO[0005] Starting agents...
INFO[0005] Starting Node 'k3d-kubefirst-agent-0'
^C
❯ k3d cluster delete kubefirst
ERRO[0000] error getting loadbalancer config from k3d-kubefirst-serverlb: runtime failed to read loadbalancer config '/etc/confd/values.yaml' from node 'k3d-kubefirst-serverlb': Error response from daemon: Could not find the file /etc/confd/values.yaml in container 874d0d1b120593e5e7cfcd852e5536d1230ff50a3b7b4fe7181d02a7db1d304d: file not found
INFO[0000] Deleting cluster 'kubefirst'
INFO[0001] Deleting cluster network 'k3d-kubefirst'
INFO[0001] Deleting 1 attached volumes...
INFO[0001] Removing cluster details from default kubeconfig...
INFO[0001] Removing standalone kubeconfig file (if there is one)...
INFO[0001] Successfully deleted cluster kubefirst!
❯ /root/.k1/kubefirst/tools/k3d cluster create kubefirst --image rancher/k3s:v1.26.3-k3s1 --agents 1 --registry-create k3d-kubefirst-registry --volume /root/.k1/kubefirst/minio-storage:/var/lib/rancher/k3s/storage@all --port 443:443@loadbalancer --verbose
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] Runtime Info:
&{Name:docker Endpoint:/var/run/docker.sock Version:20.10.21 OSType:linux OS:Rancher Desktop WSL Distribution Arch:x86_64 CgroupVersion:1 CgroupDriver:cgroupfs Filesystem:extfs InfoName:LaptopStudio}
DEBU[0000] Additional CLI Configuration:
cli:
  api-port: ""
  env: []
  k3s-node-labels: []
  k3sargs: []
  ports:
  - 443:443@loadbalancer
  registries:
    create: k3d-kubefirst-registry
  runtime-labels: []
  volumes:
  - /root/.k1/kubefirst/minio-storage:/var/lib/rancher/k3s/storage@all
hostaliases: []
DEBU[0000] Configuration:
agents: 1
image: rancher/k3s:v1.26.3-k3s1
network: ""
options:
  k3d:
    disableimagevolume: false
    disableloadbalancer: false
    disablerollback: false
    loadbalancer:
      configoverrides: []
    timeout: 0s
    wait: true
  kubeconfig:
    switchcurrentcontext: true
    updatedefaultkubeconfig: true
  runtime:
    agentsmemory: ""
    gpurequest: ""
    hostpidmode: false
    serversmemory: ""
registries:
  config: ""
  use: []
servers: 1
subnet: ""
token: ""
DEBU[0000] ========== Simple Config ==========
{TypeMeta:{Kind:Simple APIVersion:k3d.io/v1alpha4} ObjectMeta:{Name:} Servers:1 Agents:1 ExposeAPI:{Host: HostIP: HostPort:} Image:rancher/k3s:v1.26.3-k3s1 Network: Subnet: ClusterToken: Volumes:[] Ports:[] Options:{K3dOptions:{Wait:true Timeout:0s DisableLoadbalancer:false DisableImageVolume:false NoRollback:false NodeHookActions:[] Loadbalancer:{ConfigOverrides:[]}} K3sOptions:{ExtraArgs:[] NodeLabels:[]} KubeconfigOptions:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true} Runtime:{GPURequest: ServersMemory: AgentsMemory: HostPidMode:false Labels:[]}} Env:[] Registries:{Use:[] Create:<nil> Config:} HostAliases:[]}
==========================
DEBU[0000] ========== Merged Simple Config ==========
{TypeMeta:{Kind:Simple APIVersion:k3d.io/v1alpha4} ObjectMeta:{Name:} Servers:1 Agents:1 ExposeAPI:{Host: HostIP: HostPort:38873} Image:rancher/k3s:v1.26.3-k3s1 Network: Subnet: ClusterToken: Volumes:[{Volume:/root/.k1/kubefirst/minio-storage:/var/lib/rancher/k3s/storage NodeFilters:[all]}] Ports:[{Port:443:443 NodeFilters:[loadbalancer]}] Options:{K3dOptions:{Wait:true Timeout:0s DisableLoadbalancer:false DisableImageVolume:false NoRollback:false NodeHookActions:[] Loadbalancer:{ConfigOverrides:[]}} K3sOptions:{ExtraArgs:[] NodeLabels:[]} KubeconfigOptions:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true} Runtime:{GPURequest: ServersMemory: AgentsMemory: HostPidMode:false Labels:[]}} Env:[] Registries:{Use:[] Create:0xc00029a000 Config:} HostAliases:[]}
==========================
INFO[0000] portmapping '443:443' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy]
DEBU[0000] generated loadbalancer config:
ports:
  443.tcp:
  - k3d-kubefirst-server-0
  - k3d-kubefirst-agent-0
  6443.tcp:
  - k3d-kubefirst-server-0
settings:
  workerConnections: 1024
DEBU[0000] Port Exposure Mapping didn't specify hostPort, choosing one randomly...
DEBU[0000] Got free port for Port Exposure: '44361'
DEBU[0000] ===== Merged Cluster Config =====
&{TypeMeta:{Kind: APIVersion:} Cluster:{Name:kubefirst Network:{Name:k3d-kubefirst ID: External:false IPAM:{IPPrefix:zero IPPrefix IPsUsed:[] Managed:false} Members:[]} Token: Nodes:[0xc0005a2ea0 0xc0005a3040 0xc0005a31e0] InitNode:<nil> ExternalDatastore:<nil> KubeAPI:0xc0006a1940 ServerLoadBalancer:0xc000226cd0 ImageVolume: Volumes:[]} ClusterCreateOpts:{DisableImageVolume:false WaitForServer:true Timeout:0s DisableLoadBalancer:false GPURequest: ServersMemory: AgentsMemory: NodeHooks:[] GlobalLabels:map[app:k3d] GlobalEnv:[] HostAliases:[] Registries:{Create:0xc0000341e0 Use:[] Config:<nil>}} KubeconfigOpts:{UpdateDefaultKubeconfig:true SwitchCurrentContext:true}}
===== ===== =====
DEBU[0000] '--kubeconfig-update-default set: enabling wait-for-server
INFO[0000] Prep: Network
INFO[0000] Created network 'k3d-kubefirst'
INFO[0000] Created image volume k3d-kubefirst-images
INFO[0000] Creating node 'k3d-kubefirst-registry'
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] Created container k3d-kubefirst-registry (ID: 475022ae93fd23e915bd1a6dfa43430ae0d222be955523d78d10af1dafae0a91)
INFO[0000] Successfully created registry 'k3d-kubefirst-registry'
DEBU[0000] no netlabel present on container /k3d-kubefirst-registry
DEBU[0000] failed to get IP for container /k3d-kubefirst-registry as we couldn't find the cluster network
DEBU[0000] no netlabel present on container /k3d-kubefirst-registry
DEBU[0000] failed to get IP for container /k3d-kubefirst-registry as we couldn't find the cluster network
INFO[0000] Starting new tools node...
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] DOCKER_SOCK=/var/run/docker.sock
DEBU[0000] [Docker] DockerHost: '' ()
DEBU[0000] Created container k3d-kubefirst-tools (ID: 38efb0c614af4276f9f94d10fcc8666b042944f1d2e82125fa54dcbd4101169a)
DEBU[0000] Node k3d-kubefirst-tools Start Time: 2023-05-23 21:58:29.528190044 -0500 CDT m=+0.121665357
INFO[0000] Starting Node 'k3d-kubefirst-tools'
DEBU[0000] Truncated 2023-05-24 02:58:29.886200305 +0000 UTC to 2023-05-24 02:58:29 +0000 UTC
INFO[0001] Creating node 'k3d-kubefirst-server-0'
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
DEBU[0001] Created container k3d-kubefirst-server-0 (ID: fd0b01cbe3da49c68974a19597c1287e92eb893beffbd57d9a5156d2d1b4c036)
DEBU[0001] Created node 'k3d-kubefirst-server-0'
INFO[0001] Creating node 'k3d-kubefirst-agent-0'
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
DEBU[0001] Created container k3d-kubefirst-agent-0 (ID: 174b7d47cdbb0d9ee0d5f7aa7d67e0f9730e34ad14734320af1be88e113c6aa9)
DEBU[0001] Created node 'k3d-kubefirst-agent-0'
INFO[0001] Creating LoadBalancer 'k3d-kubefirst-serverlb'
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
DEBU[0001] Created container k3d-kubefirst-serverlb (ID: 83710d2413ac2324e165666c1e80c77de2749b1d6baaa70d73f9f87041c51b70)
DEBU[0001] Created loadbalancer 'k3d-kubefirst-serverlb'
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
INFO[0001] Using the k3d-tools node to gather environment information
DEBU[0001] no netlabel present on container /k3d-kubefirst-tools
DEBU[0001] failed to get IP for container /k3d-kubefirst-tools as we couldn't find the cluster network
DEBU[0001] Deleting node k3d-kubefirst-tools ...
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
INFO[0001] HostIP: using network gateway 172.31.0.1 address
INFO[0001] Starting cluster 'kubefirst'
INFO[0001] Starting servers...
DEBU[0001] DOCKER_SOCK=/var/run/docker.sock
DEBU[0001] No fix enabled.
DEBU[0001] Node k3d-kubefirst-server-0 Start Time: 2023-05-23 21:58:31.006247299 -0500 CDT m=+1.599722627
INFO[0001] Starting Node 'k3d-kubefirst-server-0'
DEBU[0002] Truncated 2023-05-24 02:58:31.414452587 +0000 UTC to 2023-05-24 02:58:31 +0000 UTC
DEBU[0002] Waiting for node k3d-kubefirst-server-0 to get ready (Log: 'k3s is up and running')
DEBU[0005] Finished waiting for log message 'k3s is up and running' from node 'k3d-kubefirst-server-0'
INFO[0005] Starting agents...
DEBU[0005] DOCKER_SOCK=/var/run/docker.sock
DEBU[0005] No fix enabled.
DEBU[0005] Node k3d-kubefirst-agent-0 Start Time: 2023-05-23 21:58:34.766487856 -0500 CDT m=+5.359963170
INFO[0005] Starting Node 'k3d-kubefirst-agent-0'
DEBU[0005] Truncated 2023-05-24 02:58:35.207079775 +0000 UTC to 2023-05-24 02:58:35 +0000 UTC
DEBU[0005] Waiting for node k3d-kubefirst-agent-0 to get ready (Log: 'Successfully registered node')

claywd avatar May 24 '23 03:05 claywd

Same issue on this version: k3d --version k3d version v5.6.0 k3s version v1.27.4-k3s1 (default) In addition I use podman 4.6.2 in rootless mode on Linux Mint (Ubuntu). Creating a cluster without specifying --agents works. If I specify --agents cluster creation hangs after creation of the agents. Same with adding an agent node after successfully created a cluster.

Is there a chance that this will work in the future?

2fxprogeeme avatar Dec 17 '23 17:12 2fxprogeeme

Same issue as @habibutsu on mac m1 14.2.1 k3d version v5.6.0 k3s version v1.27.5-k3s1 (default)

luxor37 avatar Jan 11 '24 21:01 luxor37

In case this helps anyone, I was using a different default-runtime in my /etc/docker/daemon.json, switching back to the default runc (by removing my previously set override) fixed the issue for me.

noamsan avatar Mar 07 '24 10:03 noamsan

I had this issue when I installed k3d from Homebrew. I think it's because Homebrew doesn't have proper access to docker and can't use docker. so when I run k3d with sudo it's worked.

amirshabanics avatar Apr 15 '24 09:04 amirshabanics