lima icon indicating copy to clipboard operation
lima copied to clipboard

vmnet: Support socket_vmnet; deprecate vde_vmnet

Open AkihiroSuda opened this issue 3 years ago • 9 comments

socket_vmnet is similar to vde_vmnet but does not depend on VDE.

https://github.com/lima-vm/socket_vmnet

AkihiroSuda avatar May 10 '22 10:05 AkihiroSuda

cc @abiosoft

AkihiroSuda avatar May 10 '22 10:05 AkihiroSuda

Will Lima be able to use unmanaged socket_vmnet like the deprecated vde_vmnet?

abiosoft avatar May 10 '22 13:05 abiosoft

Will Lima be able to use unmanaged socket_vmnet like the deprecated vde_vmnet?

I have only briefly looked at this PR, and it doesn't seem to be implemented.

I will be interested in unmanged vmnet as well, as experience with Rancher Desktop has shown that the whole sudoers mechanism is problematic for a significant subset of users, so requiring admin rights only once during installation is preferable.

I'll do a review of this PR later to see if adding unmanaged socket_vmnet support seems feasible.

jandubois avatar May 10 '22 17:05 jandubois

I will be interested in unmanged vmnet as well, as experience with Rancher Desktop has shown that the whole sudoers mechanism is problematic for a significant subset of users, so requiring admin rights only once during installation is preferable.

This is exactly the same reason I am interested in unmanaged vmnet. :D

I have only briefly looked at this PR, and it doesn't seem to be implemented.

I had a look through the code as well and noticed it is missing. I asked the question to ascertain if that is due to the PR being a WIP or a deliberate omission.

abiosoft avatar May 10 '22 18:05 abiosoft

I can update PR to support unmanaged socks (by the end of this week or maybe next week)

AkihiroSuda avatar May 11 '22 17:05 AkihiroSuda

I can update PR to support unmanaged socks (by the end of this week or maybe next week)

That would be great; could also be a follow-up PR because the current PR doesn't remove the ability to use unmanaged vde_vmnet daemons; it only switches the managed networks from vde_vmnet to socket_vmnet.

jandubois avatar May 11 '22 17:05 jandubois

Updated the PR to support unmanaged socks

networks:
  # Lima can also connect to "unmanaged" networks addressed by "socket". This
  # means that the daemons will not be controlled by Lima, but must be started
  # before the instance.  The interface type (host, shared, or bridged) is
  # configured in socket_vmnet and not in lima.
  # - socket: "/var/run/socket_vmnet"

Currently this requires socket_vmnet_client in the PATH.

AkihiroSuda avatar May 16 '22 16:05 AkihiroSuda

iperf3 benchmark (host -> guest)

  • vde_vmnet: 321 Mbps / 318 Mbps
  • socket_vmnet: 686 Mbps / 686 Mbps

vde_vmnet

$ iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[  5] local 192.168.105.1 port 62788 connected to 192.168.105.3 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  42.6 MBytes   357 Mbits/sec                  
[  5]   1.00-2.00   sec  39.7 MBytes   333 Mbits/sec                  
[  5]   2.00-3.00   sec  39.3 MBytes   329 Mbits/sec                  
[  5]   3.00-4.00   sec  37.8 MBytes   317 Mbits/sec                  
[  5]   4.00-5.00   sec  37.2 MBytes   312 Mbits/sec                  
[  5]   5.00-6.00   sec  37.2 MBytes   312 Mbits/sec                  
[  5]   6.00-7.00   sec  37.8 MBytes   317 Mbits/sec                  
[  5]   7.00-8.00   sec  38.4 MBytes   322 Mbits/sec                  
[  5]   8.00-9.00   sec  35.5 MBytes   298 Mbits/sec                  
[  5]   9.00-10.00  sec  36.9 MBytes   309 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   382 MBytes   321 Mbits/sec                  sender
[  5]   0.00-10.05  sec   381 MBytes   318 Mbits/sec                  receiver

iperf Done.

socket_vmnet

$ iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[  5] local 192.168.105.1 port 62814 connected to 192.168.105.3 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  85.2 MBytes   715 Mbits/sec                  
[  5]   1.00-2.00   sec  82.7 MBytes   694 Mbits/sec                  
[  5]   2.00-3.00   sec  81.7 MBytes   686 Mbits/sec                  
[  5]   3.00-4.00   sec  82.3 MBytes   690 Mbits/sec                  
[  5]   4.00-5.00   sec  82.0 MBytes   688 Mbits/sec                  
[  5]   5.00-6.00   sec  81.5 MBytes   684 Mbits/sec                  
[  5]   6.00-7.00   sec  79.8 MBytes   669 Mbits/sec                  
[  5]   7.00-8.00   sec  82.2 MBytes   690 Mbits/sec                  
[  5]   8.00-9.00   sec  79.4 MBytes   666 Mbits/sec                  
[  5]   9.00-10.00  sec  80.9 MBytes   678 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   818 MBytes   686 Mbits/sec                  sender
[  5]   0.00-9.99   sec   817 MBytes   686 Mbits/sec                  receiver

iperf Done.

AkihiroSuda avatar May 16 '22 16:05 AkihiroSuda

Rebased, will try to address the review comments next week

AkihiroSuda avatar Jul 21 '22 09:07 AkihiroSuda

This should be reviewable/mergeable now.

cc @jandubois @abiosoft @mook-as

AkihiroSuda avatar Aug 18 '22 15:08 AkihiroSuda

Planning to cut the next release of Lima soon. Can we merge this? Compatibility with the legacy vde_vmnet is tested in the CI.

AkihiroSuda avatar Sep 04 '22 23:09 AkihiroSuda

Planning to cut the next release of Lima soon. Can we merge this? Compatibility with the legacy vde_vmnet is tested in the CI.

I will not be able to properly test this for the next few days as I am occupied with some other things at the moment. I should give my feedback by Friday/Saturday (if that is not too late).

abiosoft avatar Sep 05 '22 03:09 abiosoft

I hope to be able to test this by Wednesday.

jandubois avatar Sep 05 '22 06:09 jandubois

I haven't looked at the code changes yet, but I just started with a quick test, creating an Alpine VM with both a shared and a bridged interface:

$ grep -A 2 networks: ~/.lima/alpine/lima.yaml
networks:
- lima: shared
  lima: bridged

I had deleted the networks.yaml to let Lima create a new file with the new default content:

$ grep -A 9 networks: ~/.lima/_config/networks.yaml
networks:
  shared:
    mode: shared
    gateway: 192.168.105.1
    dhcpEnd: 192.168.105.254
    netmask: 255.255.255.0
  bridged:
    mode: bridged
    interface: en0
    # bridged mode doesn't have a gateway; dhcp is managed by outside network

But both interfaces are created in the DHCP range of the bridged network (192.168.18.0/24):

$ limactl shell alpine ip a | grep -A 2 lima0:
3: lima0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:55:55:3d:01:95 brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.111/16 scope global lima0
$ limactl shell alpine ip a | grep -A 2 lima1:
4: lima1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:55:55:9b:53:e4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.160/16 scope global lima1

I also see just a single socket_vmnet daemon process for bridged networking, and none for the shared network.

So this is still a regression to the vde_vmnet implementation.

jandubois avatar Sep 06 '22 23:09 jandubois

$ grep -A 2 networks: ~/.lima/alpine/lima.yaml
networks:
- lima: shared
  lima: bridged

Probably you meant this:

networks:
- lima: shared
- lima: bridged

(And it works for me)

AkihiroSuda avatar Sep 07 '22 03:09 AkihiroSuda

Probably you meant this:

networks:
- lima: shared
- lima: bridged

Indeed, I did. I didn't notice because I still got 2 networks. Which happened because I had defined a bridged network in my default.yaml, but had forgotten about...

(And it works for me)

Yes, works for me too now! Thanks!

Will try to do another code review tomorrow.

jandubois avatar Sep 07 '22 05:09 jandubois

Will try to do another code review tomorrow.

I have not finished this today; will try to wrap up tomorrow. So far LGTM.

jandubois avatar Sep 08 '22 05:09 jandubois

Rebased

AkihiroSuda avatar Sep 08 '22 20:09 AkihiroSuda

I am probably late to the party already.

Nonetheless, I've finally got the time to port Colima to this and test properly.

Thanks for the hardwork.

abiosoft avatar Sep 18 '22 10:09 abiosoft