socket_vmnet icon indicating copy to clipboard operation
socket_vmnet copied to clipboard

Published 20 hours ago • verified publisher icon lima-vm

DHCP leases file not getting updated

Open spowelljr opened this issue 3 years ago • 49 comments

We've been getting issues on our personal machines where our DHCP leases file is not being updated after starting an instance with socket_vmnet. The fix so far has been uninstall socket_vmnet, reboot machine, reinstall, which resolves the issue. However I'm trying to integrate GitHub Action tests for QEMU/socket_vmnet and I'm getting the same thing (logs below).

Is there a command we could run that would resolve this issue? I see in the README there's a command to Reload the DHCP daemon

sudo /bin/launchctl kickstart -kp system/com.apple.bootpd

I'm not sure if that should resolve the issue, however trying to run it in GitHub Action resulted in Could not find service "com.apple.bootpd" in domain for system

Below are to logs from the GitHub Action machine with the main error being StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory

We experienced the same thing on our personal machines, we tried manually creating the file just incase the there were some permission errors but that didn't resolve the issue, plus I'm sure it's more than capable of creating the file itself.

Any help would be appreciated, thanks!

functional_test.go:2161: (dbg) Run:  ./minikube-darwin-amd64 start -p func-184505 --memory=4000 --apiserver-port=8441 --wait=all --driver=qemu
    functional_test.go:2161: (dbg) Non-zero exit: ./minikube-darwin-amd64 start -p func-184505 --memory=4000 --apiserver-port=8441 --wait=all --driver=qemu: exit status 80 (2m23.88093201s)
        
        -- stdout --
        	* [func-184505] minikube v1.28.0 on Darwin 12.6.1
        	  - MINIKUBE_HOME=/Users/runner/work/minikube/minikube/minikube_binaries/testhome
        	  - KUBECONFIG=/Users/runner/work/minikube/minikube/minikube_binaries/testhome/kubeconfig
        	* Using the qemu2 (experimental) driver based on user configuration
        	* Automatically selected the socket_vmnet network
        	* Downloading VM boot image ...
        	* Starting control plane node func-184505 in cluster func-184505
        	* Downloading Kubernetes v1.25.3 preload ...
        	* Creating qemu2 VM (CPUs=2, Memory=[40](https://github.com/spowelljr/minikube/actions/runs/3604165929/jobs/6073444435#step:12:41)00MB, Disk=20000MB) ...
        	* Deleting "func-184505" in qemu2 ...
        	* Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
        	
        	
        
        -- /stdout --
        ** stderr ** 
        	! Local proxy ignored: not passing HTTP_PROXY=localhost:49253 to docker env.
        	! Local proxy ignored: not passing HTTP_PROXY=localhost:49253 to docker env.
        	! StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory
        	! Local proxy ignored: not passing HTTP_PROXY=localhost:49253 to docker env.
        	* Failed to start qemu2 VM. Running "minikube delete -p func-18[45](https://github.com/spowelljr/minikube/actions/runs/3604165929/jobs/6073444435#step:12:46)05" may fix it: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory
        	X Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory

spowelljr avatar Dec 03 '22 00:12 spowelljr

any idea @AkihiroSuda what might cause this? we see this error as a Flake on our mac Os sometimes, but on Github Action it happens all the time. so I think finding the root cause would help a lot of users

medyagh avatar Dec 16 '22 17:12 medyagh

Could you make sure that you are NOT using the bridged mode?

Github Action

Bridged mode is known not to work on Github Action

AkihiroSuda avatar Dec 16 '22 18:12 AkihiroSuda

I'm just running

sudo make PREFIX=/opt/socket_vmnet install

Which according to the comments in the Makefile Empty value (default) disables bridged mode. bridged mode is not being used.

spowelljr avatar Dec 16 '22 23:12 spowelljr

@AkihiroSuda are there any debugging commands you suggest I could run to get you more logs? I currently have the problem on Mac M1 with minikube here are the logs if it helps with debugging (the log is too long I attach here) lastStart.txt

but here is the last part of the log


I0103 14:33:10.388012    8417 main.go:134] libmachine: STDERR: 
I0103 14:33:10.388025    8417 main.go:134] libmachine: DONE writing to /Users/medya/.minikube/machines/minikube/disk.qcow2.raw and /Users/medya/.minikube/machines/minikube/disk.qcow2
I0103 14:33:10.388028    8417 main.go:134] libmachine: Starting QEMU VM...
I0103 14:33:10.388082    8417 main.go:134] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/Cellar/qemu/7.1.0/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/medya/.minikube/machines/minikube/boot2docker.iso -qmp unix:/Users/medya/.minikube/machines/minikube/monitor,server,nowait -pidfile /Users/medya/.minikube/machines/minikube/qemu.pid -device virtio-net-pci,netdev=net0,mac=6e:a5:b1:18:4b:c6 -netdev socket,id=net0,fd=3 -daemonize /Users/medya/.minikube/machines/minikube/disk.qcow2
I0103 14:33:10.554509    8417 main.go:134] libmachine: STDOUT: 
I0103 14:33:10.554525    8417 main.go:134] libmachine: STDERR: 
I0103 14:33:10.554530    8417 main.go:134] libmachine: Attempt 0
I0103 14:33:10.554541    8417 main.go:134] libmachine: Searching for 6e:a5:b1:18:4b:c6 in /var/db/dhcpd_leases ...
I0103 14:33:10.555184    8417 main.go:134] libmachine: Found 29 entries in /var/db/dhcpd_leases!
I0103 14:33:10.555206    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.30 HWAddress:be:cf:36:32:32:50 ID:1,be:cf:36:32:32:50 Lease:0x638fe0ed}
I0103 14:33:10.555218    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.29 HWAddress:fa:f0:51:fe:f7:1a ID:1,fa:f0:51:fe:f7:1a Lease:0x638bb50d}
I0103 14:33:10.555223    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.28 HWAddress:6e:89:68:eb:29:5d ID:1,6e:89:68:eb:29:5d Lease:0x638bb494}
I0103 14:33:10.555229    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.27 HWAddress:ca:fb:db:5f:2d:24 ID:1,ca:fb:db:5f:2d:24 Lease:0x638bb19f}
I0103 14:33:10.555242    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.26 HWAddress:d2:3e:a3:7b:7d:f2 ID:1,d2:3e:a3:7b:7d:f2 Lease:0x638ba52c}
I0103 14:33:10.555248    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.25 HWAddress:6e:35:16:67:6f:a8 ID:1,6e:35:16:67:6f:a8 Lease:0x638b9adf}
I0103 14:33:10.555253    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.24 HWAddress:fa:d8:27:37:78:5a ID:1,fa:d8:27:37:78:5a Lease:0x638b9788}
I0103 14:33:10.555259    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.23 HWAddress:46:94:f8:3a:f9:c1 ID:1,46:94:f8:3a:f9:c1 Lease:0x638a7665}
I0103 14:33:10.555266    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.22 HWAddress:6:ac:ca:d6:bc:d0 ID:1,6:ac:ca:d6:bc:d0 Lease:0x6374114f}
I0103 14:33:10.555271    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.21 HWAddress:f2:b0:8:d2:81:81 ID:1,f2:b0:8:d2:81:81 Lease:0x637560c7}
I0103 14:33:10.555276    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.20 HWAddress:56:4b:62:8e:98:7 ID:1,56:4b:62:8e:98:7 Lease:0x63753f29}
I0103 14:33:10.555281    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.19 HWAddress:e6:86:66:d0:9e:39 ID:1,e6:86:66:d0:9e:39 Lease:0x636ec84f}
I0103 14:33:10.555287    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.18 HWAddress:6a:18:75:f:19:7a ID:1,6a:18:75:f:19:7a Lease:0x636ec348}
I0103 14:33:10.555292    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.17 HWAddress:26:34:18:84:98:cb ID:1,26:34:18:84:98:cb Lease:0x636eb11c}
I0103 14:33:10.555307    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.16 HWAddress:ee:25:74:7a:65:f6 ID:1,ee:25:74:7a:65:f6 Lease:0x636eb03f}
I0103 14:33:10.555313    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.15 HWAddress:9a:70:93:66:fa:b5 ID:1,9a:70:93:66:fa:b5 Lease:0x636ea065}
I0103 14:33:10.555318    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.14 HWAddress:96:25:4e:67:c7:fb ID:1,96:25:4e:67:c7:fb Lease:0x636ea026}
I0103 14:33:10.555324    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.13 HWAddress:3a:19:ee:fb:ee:a2 ID:1,3a:19:ee:fb:ee:a2 Lease:0x636e9bc8}
I0103 14:33:10.555333    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.12 HWAddress:3a:dd:b7:8c:e0:7f ID:1,3a:dd:b7:8c:e0:7f Lease:0x636e9519}
I0103 14:33:10.555339    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.11 HWAddress:d2:4e:50:2c:3f:9d ID:1,d2:4e:50:2c:3f:9d Lease:0x636d910e}
I0103 14:33:10.555344    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.10 HWAddress:e2:e5:a7:65:9e:fd ID:1,e2:e5:a7:65:9e:fd Lease:0x636d8e99}
I0103 14:33:10.555349    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.9 HWAddress:e:75:98:a:a4:5b ID:1,e:75:98:a:a4:5b Lease:0x636d8a61}
I0103 14:33:10.555354    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.8 HWAddress:7a:f3:e3:a6:e5:42 ID:1,7a:f3:e3:a6:e5:42 Lease:0x636d88fd}
I0103 14:33:10.555359    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.7 HWAddress:3a:36:74:65:54:48 ID:1,3a:36:74:65:54:48 Lease:0x636d7a5c}
I0103 14:33:10.555365    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.6 HWAddress:be:1a:aa:7c:bb:71 ID:1,be:1a:aa:7c:bb:71 Lease:0x636d7183}
I0103 14:33:10.555370    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.5 HWAddress:c6:c1:8d:ef:8c:6a ID:1,c6:c1:8d:ef:8c:6a Lease:0x636d7108}
I0103 14:33:10.555375    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:c2:79:20:b8:68:de ID:1,c2:79:20:b8:68:de Lease:0x636d3d9c}
I0103 14:33:10.555390    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:76:f5:fd:cf:d6:1 ID:1,76:f5:fd:cf:d6:1 Lease:0x636c6368}
I0103 14:33:10.555395    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:6a:c5:89:c1:b8:15 ID:1,6a:c5:89:c1:b8:15 Lease:0x636c6200}
I0103 14:33:12.556467    8417 main.go:134] libmachine: Attempt 1
I0103 14:33:12.556499    8417 main.go:134] libmachine: Searching for 6e:a5:b1:18:4b:c6 in /var/db/dhcpd_leases ...
I0103 14:33:12.556706    8417 main.go:134] libmachine: Found 29 entries in /var/db/dhcpd_leases!
I0103 14:33:12.556731    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.30 HWAddress:be:cf:36:32:32:50 ID:1,be:cf:36:32:32:50 Lease:0x638fe0ed}
I0103 14:33:12.556738    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.29 HWAddress:fa:f0:51:fe:f7:1a ID:1,fa:f0:51:fe:f7:1a Lease:0x638bb50d}
I0103 14:33:12.556744    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.28 HWAddress:6e:89:68:eb:29:5d ID:1,6e:89:68:eb:29:5d Lease:0x638bb494}
I0103 14:33:12.556751    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.27 HWAddress:ca:fb:db:5f:2d:24 ID:1,ca:fb:db:5f:2d:24 Lease:0x638bb19f}
I0103 14:33:12.556757    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.26 HWAddress:d2:3e:a3:7b:7d:f2 ID:1,d2:3e:a3:7b:7d:f2 Lease:0x638ba52c}
I0103 14:33:12.556763    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.25 HWAddress:6e:35:16:67:6f:a8 ID:1,6e:35:16:67:6f:a8 Lease:0x638b9adf}
I0103 14:33:12.556769    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.24 HWAddress:fa:d8:27:37:78:5a ID:1,fa:d8:27:37:78:5a Lease:0x638b9788}
I0103 14:33:12.556775    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.23 HWAddress:46:94:f8:3a:f9:c1 ID:1,46:94:f8:3a:f9:c1 Lease:0x638a7665}
I0103 14:33:12.556783    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.22 HWAddress:6:ac:ca:d6:bc:d0 ID:1,6:ac:ca:d6:bc:d0 Lease:0x6374114f}
I0103 14:33:12.556789    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.21 HWAddress:f2:b0:8:d2:81:81 ID:1,f2:b0:8:d2:81:81 Lease:0x637560c7}
I0103 14:33:12.556795    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.20 HWAddress:56:4b:62:8e:98:7 ID:1,56:4b:62:8e:98:7 Lease:0x63753f29}
I0103 14:33:12.556801    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.19 HWAddress:e6:86:66:d0:9e:39 ID:1,e6:86:66:d0:9e:39 Lease:0x636ec84f}
I0103 14:33:12.556806    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.18 HWAddress:6a:18:75:f:19:7a ID:1,6a:18:75:f:19:7a Lease:0x636ec348}
I0103 14:33:12.556812    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.17 HWAddress:26:34:18:84:98:cb ID:1,26:34:18:84:98:cb Lease:0x636eb11c}
I0103 14:33:12.556819    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.16 HWAddress:ee:25:74:7a:65:f6 ID:1,ee:25:74:7a:65:f6 Lease:0x636eb03f}
I0103 14:33:12.556825    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.15 HWAddress:9a:70:93:66:fa:b5 ID:1,9a:70:93:66:fa:b5 Lease:0x636ea065}
I0103 14:33:12.556831    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.14 HWAddress:96:25:4e:67:c7:fb ID:1,96:25:4e:67:c7:fb Lease:0x636ea026}
I0103 14:33:12.556837    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.13 HWAddress:3a:19:ee:fb:ee:a2 ID:1,3a:19:ee:fb:ee:a2 Lease:0x636e9bc8}
I0103 14:33:12.556843    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.12 HWAddress:3a:dd:b7:8c:e0:7f ID:1,3a:dd:b7:8c:e0:7f Lease:0x636e9519}
I0103 14:33:12.556852    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.11 HWAddress:d2:4e:50:2c:3f:9d ID:1,d2:4e:50:2c:3f:9d Lease:0x636d910e}
I0103 14:33:12.556858    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.10 HWAddress:e2:e5:a7:65:9e:fd ID:1,e2:e5:a7:65:9e:fd Lease:0x636d8e99}
I0103 14:33:12.556864    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.9 HWAddress:e:75:98:a:a4:5b ID:1,e:75:98:a:a4:5b Lease:0x636d8a61}
I0103 14:33:12.556870    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.8 HWAddress:7a:f3:e3:a6:e5:42 ID:1,7a:f3:e3:a6:e5:42 Lease:0x636d88fd}
I0103 14:33:12.556876    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.7 HWAddress:3a:36:74:65:54:48 ID:1,3a:36:74:65:54:48 Lease:0x636d7a5c}
I0103 14:33:12.556882    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.6 HWAddress:be:1a:aa:7c:bb:71 ID:1,be:1a:aa:7c:bb:71 Lease:0x636d7183}
I0103 14:33:12.556888    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.5 HWAddress:c6:c1:8d:ef:8c:6a ID:1,c6:c1:8d:ef:8c:6a Lease:0x636d7108}
I0103 14:33:12.556894    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:c2:79:20:b8:68:de ID:1,c2:79:20:b8:68:de Lease:0x636d3d9c}
I0103 14:33:12.556900    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:76:f5:fd:cf:d6:1 ID:1,76:f5:fd:cf:d6:1 Lease:0x636c6368}
I0103 14:33:12.556906    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:6a:c5:89:c1:b8:15 ID:1,6a:c5:89:c1:b8:15 Lease:0x636c6200}
I0103 14:33:14.557930    8417 main.go:134] libmachine: Attempt 2
I0103 14:33:14.557940    8417 main.go:134] libmachine: Searching for 6e:a5:b1:18:4b:c6 in /var/db/dhcpd_leases ...
I0103 14:33:14.558076    8417 main.go:134] libmachine: Found 29 entries in /var/db/dhcpd_leases!
I0103 14:33:14.558084    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.30 HWAddress:be:cf:36:32:32:50 ID:1,be:cf:36:32:32:50 Lease:0x638fe0ed}
I0103 14:33:14.558088    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.29 HWAddress:fa:f0:51:fe:f7:1a ID:1,fa:f0:51:fe:f7:1a Lease:0x638bb50d}
I0103 14:33:14.558093    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.28 HWAddress:6e:89:68:eb:29:5d ID:1,6e:89:68:eb:29:5d Lease:0x638bb494}
I0103 14:33:14.558097    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.27 HWAddress:ca:fb:db:5f:2d:24 ID:1,ca:fb:db:5f:2d:24 Lease:0x638bb19f}
I0103 14:33:14.558101    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.26 HWAddress:d2:3e:a3:7b:7d:f2 ID:1,d2:3e:a3:7b:7d:f2 Lease:0x638ba52c}
I0103 14:33:14.558104    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.25 HWAddress:6e:35:16:67:6f:a8 ID:1,6e:35:16:67:6f:a8 Lease:0x638b9adf}
I0103 14:33:14.558108    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.24 HWAddress:fa:d8:27:37:78:5a ID:1,fa:d8:27:37:78:5a Lease:0x638b9788}
I0103 14:33:14.558112    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.23 HWAddress:46:94:f8:3a:f9:c1 ID:1,46:94:f8:3a:f9:c1 Lease:0x638a7665}
I0103 14:33:14.558116    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.22 HWAddress:6:ac:ca:d6:bc:d0 ID:1,6:ac:ca:d6:bc:d0 Lease:0x6374114f}
I0103 14:33:14.558129    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.21 HWAddress:f2:b0:8:d2:81:81 ID:1,f2:b0:8:d2:81:81 Lease:0x637560c7}
I0103 14:33:14.558133    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.20 HWAddress:56:4b:62:8e:98:7 ID:1,56:4b:62:8e:98:7 Lease:0x63753f29}
I0103 14:33:14.558137    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.19 HWAddress:e6:86:66:d0:9e:39 ID:1,e6:86:66:d0:9e:39 Lease:0x636ec84f}
I0103 14:33:14.558141    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.18 HWAddress:6a:18:75:f:19:7a ID:1,6a:18:75:f:19:7a Lease:0x636ec348}
I0103 14:33:14.558145    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.17 HWAddress:26:34:18:84:98:cb ID:1,26:34:18:84:98:cb Lease:0x636eb11c}
I0103 14:33:14.558149    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.16 HWAddress:ee:25:74:7a:65:f6 ID:1,ee:25:74:7a:65:f6 Lease:0x636eb03f}
I0103 14:33:14.558153    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.15 HWAddress:9a:70:93:66:fa:b5 ID:1,9a:70:93:66:fa:b5 Lease:0x636ea065}
I0103 14:33:14.558156    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.14 HWAddress:96:25:4e:67:c7:fb ID:1,96:25:4e:67:c7:fb Lease:0x636ea026}
I0103 14:33:14.558160    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.13 HWAddress:3a:19:ee:fb:ee:a2 ID:1,3a:19:ee:fb:ee:a2 Lease:0x636e9bc8}
I0103 14:33:14.558164    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.12 HWAddress:3a:dd:b7:8c:e0:7f ID:1,3a:dd:b7:8c:e0:7f Lease:0x636e9519}
I0103 14:33:14.558168    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.11 HWAddress:d2:4e:50:2c:3f:9d ID:1,d2:4e:50:2c:3f:9d Lease:0x636d910e}
I0103 14:33:14.558172    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.10 HWAddress:e2:e5:a7:65:9e:fd ID:1,e2:e5:a7:65:9e:fd Lease:0x636d8e99}
I0103 14:33:14.558179    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.9 HWAddress:e:75:98:a:a4:5b ID:1,e:75:98:a:a4:5b Lease:0x636d8a61}
I0103 14:33:14.558183    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.8 HWAddress:7a:f3:e3:a6:e5:42 ID:1,7a:f3:e3:a6:e5:42 Lease:0x636d88fd}
I0103 14:33:14.558187    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.7 HWAddress:3a:36:74:65:54:48 ID:1,3a:36:74:65:54:48 Lease:0x636d7a5c}
I0103 14:33:14.558191    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.6 HWAddress:be:1a:aa:7c:bb:71 ID:1,be:1a:aa:7c:bb:71 Lease:0x636d7183}
I0103 14:33:14.558195    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.5 HWAddress:c6:c1:8d:ef:8c:6a ID:1,c6:c1:8d:ef:8c:6a Lease:0x636d7108}
I0103 14:33:14.558199    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:c2:79:20:b8:68:de ID:1,c2:79:20:b8:68:de Lease:0x636d3d9c}
I0103 14:33:14.558203    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:76:f5:fd:cf:d6:1 ID:1,76:f5:fd:cf:d6:1 Lease:0x636c6368}
I0103 14:33:14.558207    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:6a:c5:89:c1:b8:15 ID:1,6a:c5:89:c1:b8:15 Lease:0x636c6200}
I0103 14:33:16.559250    8417 main.go:134] libmachine: Attempt 3
I0103 14:33:16.559259    8417 main.go:134] libmachine: Searching for 6e:a5:b1:18:4b:c6 in /var/db/dhcpd_leases ...
I0103 14:33:16.559410    8417 main.go:134] libmachine: Found 29 entries in /var/db/dhcpd_leases!
I0103 14:33:16.559418    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.30 HWAddress:be:cf:36:32:32:50 ID:1,be:cf:36:32:32:50 Lease:0x638fe0ed}
I0103 14:33:16.559427    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.29 HWAddress:fa:f0:51:fe:f7:1a ID:1,fa:f0:51:fe:f7:1a Lease:0x638bb50d}
I0103 14:33:16.559432    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.28 HWAddress:6e:89:68:eb:29:5d ID:1,6e:89:68:eb:29:5d Lease:0x638bb494}
I0103 14:33:16.559436    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.27 HWAddress:ca:fb:db:5f:2d:24 ID:1,ca:fb:db:5f:2d:24 Lease:0x638bb19f}
I0103 14:33:16.559440    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.26 HWAddress:d2:3e:a3:7b:7d:f2 ID:1,d2:3e:a3:7b:7d:f2 Lease:0x638ba52c}
I0103 14:33:16.559444    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.25 HWAddress:6e:35:16:67:6f:a8 ID:1,6e:35:16:67:6f:a8 Lease:0x638b9adf}
I0103 14:33:16.559448    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.24 HWAddress:fa:d8:27:37:78:5a ID:1,fa:d8:27:37:78:5a Lease:0x638b9788}
I0103 14:33:16.559459    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.23 HWAddress:46:94:f8:3a:f9:c1 ID:1,46:94:f8:3a:f9:c1 Lease:0x638a7665}
I0103 14:33:16.559464    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.22 HWAddress:6:ac:ca:d6:bc:d0 ID:1,6:ac:ca:d6:bc:d0 Lease:0x6374114f}
I0103 14:33:16.559468    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.21 HWAddress:f2:b0:8:d2:81:81 ID:1,f2:b0:8:d2:81:81 Lease:0x637560c7}
I0103 14:33:16.559474    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.20 HWAddress:56:4b:62:8e:98:7 ID:1,56:4b:62:8e:98:7 Lease:0x63753f29}
I0103 14:33:16.559478    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.19 HWAddress:e6:86:66:d0:9e:39 ID:1,e6:86:66:d0:9e:39 Lease:0x636ec84f}
I0103 14:33:16.559482    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.18 HWAddress:6a:18:75:f:19:7a ID:1,6a:18:75:f:19:7a Lease:0x636ec348}
I0103 14:33:16.559486    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.17 HWAddress:26:34:18:84:98:cb ID:1,26:34:18:84:98:cb Lease:0x636eb11c}
I0103 14:33:16.559490    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.16 HWAddress:ee:25:74:7a:65:f6 ID:1,ee:25:74:7a:65:f6 Lease:0x636eb03f}
I0103 14:33:16.559500    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.15 HWAddress:9a:70:93:66:fa:b5 ID:1,9a:70:93:66:fa:b5 Lease:0x636ea065}
I0103 14:33:16.559504    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.14 HWAddress:96:25:4e:67:c7:fb ID:1,96:25:4e:67:c7:fb Lease:0x636ea026}
I0103 14:33:16.559508    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.13 HWAddress:3a:19:ee:fb:ee:a2 ID:1,3a:19:ee:fb:ee:a2 Lease:0x636e9bc8}
I0103 14:33:16.559512    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.12 HWAddress:3a:dd:b7:8c:e0:7f ID:1,3a:dd:b7:8c:e0:7f Lease:0x636e9519}
I0103 14:33:16.559516    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.11 HWAddress:d2:4e:50:2c:3f:9d ID:1,d2:4e:50:2c:3f:9d Lease:0x636d910e}
I0103 14:33:16.559520    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.10 HWAddress:e2:e5:a7:65:9e:fd ID:1,e2:e5:a7:65:9e:fd Lease:0x636d8e99}
I0103 14:33:16.559524    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.9 HWAddress:e:75:98:a:a4:5b ID:1,e:75:98:a:a4:5b Lease:0x636d8a61}
I0103 14:33:16.559528    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.8 HWAddress:7a:f3:e3:a6:e5:42 ID:1,7a:f3:e3:a6:e5:42 Lease:0x636d88fd}
I0103 14:33:16.559532    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.7 HWAddress:3a:36:74:65:54:48 ID:1,3a:36:74:65:54:48 Lease:0x636d7a5c}
I0103 14:33:16.559536    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.6 HWAddress:be:1a:aa:7c:bb:71 ID:1,be:1a:aa:7c:bb:71 Lease:0x636d7183}
I0103 14:33:16.559540    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.5 HWAddress:c6:c1:8d:ef:8c:6a ID:1,c6:c1:8d:ef:8c:6a Lease:0x636d7108}
I0103 14:33:16.559544    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:c2:79:20:b8:68:de ID:1,c2:79:20:b8:68:de Lease:0x636d3d9c}
I0103 14:33:16.559548    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:76:f5:fd:cf:d6:1 ID:1,76:f5:fd:cf:d6:1 Lease:0x636c6368}
I0103 14:33:16.559551    8417 main.go:134] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:6a:c5:89:c1:b8:15 ID:1,6a:c5:89:c1:b8:15 Lease:0x636c6200}
I0103 14:33:18.560554    8417 main.go:134] libmachine: Attempt 4
I0103 14:33:18.560563    8417 main.go:134] libmachine: Searching for 6e:a5:b1:18:4b:c6 in /var/db/dhcpd_leases ...
I0103 14:33:18.560681    8417 main.go:134] libmachine: Found 29 entries in /var/db/dhcpd_leases!

....
.... (and it keeps going like this till failure)

medyagh avatar Jan 03 '23 23:01 medyagh

When an instance is terminated it logs the following:

$ cat /var/run/socket_vmnet.stderr
on_accept(): vmnet_return_t VMNET_INVALID_ARGUMENT
vmnet_write: Undefined error: 0

Not sure if that's expected or not.

spowelljr avatar Jan 05 '23 23:01 spowelljr

@AkihiroSuda Is there anything I could do (debug commands, etc.) to help figure out this issue? I'm still experiencing this issue.

spowelljr avatar Jan 17 '23 18:01 spowelljr

Cc @jandubois could you take a look?

AkihiroSuda avatar Jan 17 '23 23:01 AkihiroSuda

I just uninstalled, rebooted, and installed with master and it works now, not sure if recent commits fixed this

spowelljr avatar Jan 20 '23 23:01 spowelljr

@spowelljr I assume you built from source? I ask because there is a brew formula for socket_vmnet as well.

mprimeaux avatar Jan 20 '23 23:01 mprimeaux

Correct, from source

spowelljr avatar Jan 20 '23 23:01 spowelljr

Thanks. Would be good to know if it still works after surviving a reboot. Regardless, I'll perform the exact steps you did and update everyone.

mprimeaux avatar Jan 20 '23 23:01 mprimeaux

TLDR; DHCP address allocation works when not in bridged mode and directly building socket_vmnet from the latest sources. Bridged mode fails.

When compiling for bridged mode with...

sudo make PREFIX=/opt/socket_vmnet install BRIDGED=en0

...and then starting minikube with....

minikube start \
  --network socket_vmnet \
  --socket-vmnet-client-path="/opt/socket_vmnet/bin/socket_vmnet_client" \
  --socket-vmnet-path="/var/run/socket_vmnet.bridged.en0"

...results in the following failure:

😿  Failed to start qemu2 VM. Running "minikube delete" may fix it: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 76:b3:20:a2:9b:74

❌  Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 76:b3:20:a2:9b:74

Of course minikube delete does not 'fix it'.

What's interesting is I can confirm a new DHCP address is being allocated since my firewall notifies me of new device activity. However, I don't see any new addresses in /var/db/dhcp_leases. I can also confirm the MAC address of the newly allocated DHCP address lease matches the MAC address in the above error.

I've also followed this "Known Issues" workaround to no avail.

UPDATE: As another interesting data point, I am able to start Lima using the same socket_vmnet in bridged mode without any issue using the following excerpt from limactl start --name=default template://vmnet.

network:
  - lima: bridged

mprimeaux avatar Jan 21 '23 17:01 mprimeaux

However, I don't see any new addresses in /var/db/dhcp_leases.

That is expected, as bridged IP addresses are assigned by the DHCP on your local network, and are not local to your host. That is the whole point of using a bridged network: to get an externally routable IP address, so other machines can connect to your VM.

Maybe this helps:

LimaVdeVmnet

socket_vmnet works basically the same, except there is no vde_switch involved.

jandubois avatar Jan 21 '23 19:01 jandubois

@jandubois Yes. That helps quite a bit. Thanks!

mprimeaux avatar Jan 21 '23 19:01 mprimeaux

@jandubois I've attached the logs.txt from my latest attempt with bridged mode.

Given the logs are Minikube, if folks prefer, I can move this over to the linked Minikube issue 15300.

mprimeaux avatar Jan 21 '23 22:01 mprimeaux

@mprimeaux I'm not sure what you are trying to achieve. You will not be able to get the IP address of the bridged network from the host machine. You will have to run e.g. ip a show lima0 (replace with iface name of your bridged network) inside the VM.

It looks like minikube only supports looking in the local leases log, so it does not support bridged networks at the moment. Somebody will have to add it. If you don't have a slirp interface to use in addition to the bridged network, then you will have to add something to the VM setup to log the address in a place that can be inspected externally.

I tried to see if there is a way to determine the address via arp -a, but it seems complicated. The address is not present initially, so I tried running ping 192.168.255.255 (because my local net is 192.168/16), but even after running it for a minute, I don't get a ping response from my VM. I have to ping it directly, but even looping over just a /24 network and pinging each address takes considerable time.

There is probably a way to force the VM to announce itself, e.g. by incorrectly broadcasting that the MAC address belongs to the host, but then giving up when the VM replies "well, actually, that is my MAC address". But that feels rather hacky (and I haven't tried if it actually works, as the bridged interface technically does belong to the host).

In all, querying the IP address via a local slirp connection seems to be the way to do this.

jandubois avatar Jan 22 '23 00:01 jandubois

I can move this over to the linked Minikube issue 15300.

I think this is just missing functionality in minikube and don't see what could be done in the socket_vmnet repo to improve the situation, so the discussion should probably more there.

jandubois avatar Jan 22 '23 00:01 jandubois

@jandubois In short, what I'm trying to accomplish is to have the Kubernetes control plane (minikube) obtain a routable address on my host network.

mprimeaux avatar Jan 22 '23 04:01 mprimeaux

I still have this problem, I confirm that uninstalling and rebooting and re-installing from source fixes it (but after a few days it goes bad again) so the root of the problem still exists on the code on HEAD.

@AkihiroSuda is there any reason a restart would fix this ? is there a process other than "/opt/socket_vmnet/bin/socket_vmnet" that we could check for, that maybe is not killed or stoppped after installation ? that only a restart kills it ?

or is there a "lock" on a file or something like that ? I just need to find out what exactly "restart" does that uninstalling/reinstalling without restart wont do the trick....

that way we could tell our users or make minikube just do that when it happens automatically

medyagh avatar Jan 25 '23 00:01 medyagh

@medyagh It's coincidental that you mention this as this happened to me just today. I ended up rebooting to fix. Could this be due to network going to sleep or an other power option in macOS? Just a hypothesis.

mprimeaux avatar Jan 25 '23 01:01 mprimeaux

My assumption is that this is an issue of Apple's bootpd (https://github.com/apple-oss-distributions/bootp) or com.apple.NetworkSharing , but not 100% sure.

If sudo /bin/launchctl kickstart -kp system/com.apple.bootpd doesn't work, sudo /bin/launchctl kickstart -kp system/com.apple.NetworkSharing may potentially work. Restarting other services (see sudo /bin/launchctl list) might be worth trying too.

AkihiroSuda avatar Jan 26 '23 09:01 AkihiroSuda

thanks for giving somethings to try out, they dont seem to fix it but i apperciate that you try

here are the logs just in case:

$ minikube start --driver=qemu2
😄  minikube v1.29.0 on Darwin 13.2 (arm64)
✨  Using the qemu2 driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for d2:c3:e7:45:fe:7d
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...\ ^C

$ minikube delete --all
🔥  Deleting "minikube" in qemu2 ...
💀  Removed all traces of the "minikube" cluster.
🔥  Successfully deleted all profiles

$ sudo /bin/launchctl kickstart -kp system/com.apple.bootpd
Could not find service "com.apple.bootpd" in domain for system
$ sudo /bin/launchctl kickstart -kp system/com.apple.NetworkSharing
service spawned with pid: 4842

$ minikube start --driver=qemu2
😄  minikube v1.29.0 on Darwin 13.2 (arm64)
✨  Using the qemu2 driver based on user configuration
🌐  Automatically selected the socket_vmnet network
👍  Starting control plane node minikube in cluster minikube
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 5e:6c:c:b9:e3:1a
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) .../ ^C


$ sudo /bin/launchctl kickstart -kp system/com.apple.NetworkSharing
service spawned with pid: 5012
$ minikube delete --all
🔥  Deleting "minikube" in qemu2 ...
💀  Removed all traces of the "minikube" cluster.
🔥  Successfully deleted all profiles

$ sudo /bin/launchctl kickstart -kp system/com.apple.NetworkSharing
service spawned with pid: 5134
$ sudo /bin/launchctl kickstart -kp system/io.github.lima-vm.socket_vmnet
service spawned with pid: 5203
$ sudo /bin/launchctl kickstart -kp system/com.apple.NetworkSharing
service spawned with pid: 5239

$ minikube start --driver=qemu2
😄  minikube v1.29.0 on Darwin 13.2 (arm64)
✨  Using the qemu2 driver based on user configuration
🌐  Automatically selected the socket_vmnet network
👍  Starting control plane node minikube in cluster minikube
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...- OUTPUT: 
ERROR: Failed to connect to "/var/run/socket_vmnet": Permission denied


🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Permission denied: exit status 1
^C

$ minikube delete --all
💀  Removed all traces of the "minikube" cluster.
🔥  Successfully deleted all profiles

$ sudo chown root:primarygroup /var/run/socket_vmnet

$ minikube start --driver=qemu2
😄  minikube v1.29.0 on Darwin 13.2 (arm64)
✨  Using the qemu2 driver based on user configuration
🌐  Automatically selected the socket_vmnet network
👍  Starting control plane node minikube in cluster minikube
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 26:9e:e3:4:84:75
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) .../

medyagh avatar Feb 02 '23 02:02 medyagh

@AkihiroSuda I wonder if could this be related to Crop VPNs ? is there a way to add logs or detect if it is being blocked by a corp VPN?

medyagh avatar Feb 22 '23 21:02 medyagh

It seems like this problem comes back after a few days of working without any issue. At first, rebooting and re-installing from the source was working but now it's not.

onatm avatar Feb 22 '23 22:02 onatm

It seems like this problem comes back after a few days of working without any issue. At first, rebooting and re-installing from the source was working but now it's not.

@onatm I am curious are you on a Corp Network or behind a VPN ?

medyagh avatar Feb 24 '23 00:02 medyagh

@medyagh FWIW, I am not and I also experience the same issue at times and, like @onatm, after a few days it fails. As another interesting data point, this occurs when on my Mac Studio (desktop) and MacBook Pro (laptop). I've played around with power options (e.g. don't sleep and wake on network activity) but the problem still persists.

Fortunately, I've been able to recover using various techniques ranging from a reboot to uninstalling / installing socket_vmnet.

mprimeaux avatar Feb 24 '23 00:02 mprimeaux

@medyagh FWIW, I am not and I also experience the same issue at times and, like @onatm, after a few days it fails. As another interesting data point, this occurs when on my Mac Studio (desktop) and MacBook Pro (laptop). I've played around with power options (e.g. don't sleep and wake on network activity) but the problem still persists.

Fortunately, I've been able to recover using various techniques ranging from a reboot to uninstalling / installing socket_vmnet.

for me I haven't recovered yet even after uninstalling/rebooting. @mprimeaux im am curious are u using this with Lima or minikube ? and are you behind a VPN or use it a corp laptop ?

medyagh avatar Mar 01 '23 21:03 medyagh

@medyagh I use socket_vmnet for both Lima and Minikube and I run them concurrently. We build our multi-arch container images on Lima and then deploy to Minikube. No corporate VPN involved as the computers aren't corporate assets.

In many cases, what works for me is (1) stop socket_vmnet, (2) uninstall socket_vmnet, (3) reboot, (4) reinstall socket_vmnet, and (5) restart socket_vmnet. All of this is a pain to say the least.

I've been playing around with running k8s in Lima since that allows me to obtain a routable address on my local network (bridged mode) as I have other IoT devices that I'm working with against the k8s cluster. That seems fine.

Also, as another data point, Lima locks up frequently when building our multi-arch container images; but only on macOS ARM64. macOS Intel is stable. The only way to recover is to force a stop limactl stop --force and then rebuild the VM. Again, less than ideal. I haven't had time to reason through hypotheses to either prove or disprove them in search of root cause.

Let me know if I can provide any other details. Just trying to get back above water so I can reconvene on testing various issues and configurations across socket_vmnet, minikube, and lima.

mprimeaux avatar Mar 02 '23 14:03 mprimeaux

...and thanks for Lima v0.15.0, its much easier to start k8s in Lima with bridged networking enabled.

limactl start --set='.cpus = 4 | .memory = "16GiB" | .networks[0] = { "socket": "/opt/homebrew/var/run/socket_vmnet.bridged.en0"}' template://k8s

Possible before but I had to download the k8s template locally and modify it. This shortens my debug cycles with various configurations.

mprimeaux avatar Mar 02 '23 15:03 mprimeaux

Adding confirmation that +1 is experiencing the issue on Ventura 13.0 M1 Max with latest minikube, qemu, and socket_vmnet.

Minikube is fine with only qemu until needing to expose external IP's to test services with external IP's.

I installed socket_vmnet & started the service like such:

brew install socket_vmnet
brew tap homebrew/services
HOMEBREW=$(which brew) && sudo ${HOMEBREW} services start socket_vmnet

I should also note that I was on an ethernet connection & no corp proxy.

If there's anything I can do to help with troubleshooting for y'all please feel free to request.

ShockleyJE avatar Mar 14 '23 14:03 ShockleyJE