colima
colima copied to clipboard
K3S install fails when network interface has multiple addresses
Description
Running colima start --kubernetes fails with the following error:
TRAC[0016] cmd ["lima" "sh" "-c" "INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_SKIP_ENABLE=true k3s-install.sh --write-kubeconfig-mode 644 --disable=traefik --advertise-address 192.168.105.3\n192.168.105.2 --flannel-iface col0 --docker --https-listen-port 63823"]
> [INFO] Skipping k3s download and verify
> [INFO] Skipping installation of SELinux RPM
> [INFO] Skipping /usr/local/bin/kubectl symlink to k3s, already exists
> [INFO] Skipping /usr/local/bin/crictl symlink to k3s, already exists
> [INFO] Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
> [INFO] Creating killall script /usr/local/bin/k3s-killall.sh
> [INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
> [INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
> [INFO] systemd: Creating service file /etc/systemd/system/k3s.service
> sh: 2: 192.168.105.2: not found
FATA[0016] error provisioning kubernetes: exit status 127
This happens because of adding a second ip address to col0, which limautil.GetIpAddress doesn't handle very well and returns a newline delimited string of ip addresses (see the value being passed in --advertise-address above).
https://github.com/abiosoft/colima/blob/eb4a76d491836ccbe11db41097f26baaef805efd/environment/vm/lima/limautil/network.go#L37-L47
You can also see this newline in for example colima list:
PROFILE STATUS ARCH CPUS MEMORY DISK RUNTIME ADDRESS
default Running aarch64 3 8GiB 100GiB docker+k3s 192.168.105.3
192.168.105.2
Version
colima version 0.8.1 git commit: 96598cc5b64e5e9e1e64891642b91edc8ac49d16
runtime: docker arch: aarch64 client: v28.3.2 server: v27.3.1 limactl version 1.1.1
Operating System
- [ ] macOS Intel <= 13 (Ventura)
- [ ] macOS Intel >= 14 (Sonoma)
- [ ] Apple Silicon <= 13 (Ventura)
- [x] Apple Silicon >= 14 (Sonoma)
- [ ] Linux
Output of colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address: 192.168.105.3
192.168.105.2
INFO[0000] socket: unix:///$HOME/.colima/default/docker.sock
Reproduction Steps
- Create a config with a fixed ip address:
provision:
- mode: system
script: ip address add 192.168.105.2/24 brd 192.168.105.255 scope global dev col0
- Run
colima start --kubernetes --profile my-profile
Expected behaviour
It's a little tricky, because I don't know how colima could pick which address it should use as the advertised ip for k3s, but even just picking the primary would be better than crashing.
Additional context
I also tried to override --advertise-address using --k3s-arg, but the argument is not replaced and instead just tacked on to the end of the command:
$ colima start -k --k3s-arg="--disable=traefik" --k3s-arg="--advertise-address=192.168.105.2" --very-verbose
...
TRAC[0025] cmd ["lima" "sh" "-c" "INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_SKIP_ENABLE=true k3s-install.sh --write-kubeconfig-mode 644 --disable=traefik --advertise-address=192.168.105.2 --advertise-address 192.168.105.5\n192.168.105.2 --flannel-iface col0 --docker --https-listen-port 57908"]
I've filed that as a separate bug in #1363