Bridged Networking or User mode? Replace ebtables & iptables with iptables-nft to allow bridged networking.

[sickcodes]

trafficstars

Here are the two types of networking

# usermode
-netdev user,id=net0,hostfwd=tcp::10022-:22,
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:09:49:17

# bridged
-netdev tap,id=net0,ifname=tap0,script=no,downscript=no
-device vmxnet3,netdev=net0,id=net0,mac=52:54:00:AB:F8:B7

user mode

The image has the same IP as your Docker IP ip n to see neighbors QEMU cannot be reached unless you add ports in the QEMU args during "docker run" Can't expose Docker ports on a running container (AFAIK), you have to start a new image and move the installation around. You also have to edit the Launch.sh or add in arguments

bridged mode

The image will run on the container's own internal network, like 10.0.2.15 or something or 192.168.122.11x

After that, you can ssh OR docker exec into the Docker and then ssh into the QEMU image.

which is better?

Realistically, they both appear exactly the same. Bridge mode would let people forward all connections to the bridge IP.

# usermode
# you have to add
docker run ... -p 50922:10022 -e EXTRA='-netdev user,hostfwd=tcp::10022-:22'

# bridge-mode
docker run ... -p 50922:10022 -ip p.ubl.ic.ip 

If anyone wants to test it out:

sudo tee -a /etc/sysctl.conf <<< 'net.ipv4.ip_forward=1'
sudo sysctl -p
sudo pacman -S iptables-nft

sudo systemctl enable --now libvirtd.service
sudo systemctl enable --now virtlogd.service

sudo ip tuntap add dev tap0 mode tap
sudo ip link set tap0 up promisc on 

sudo virsh net-autostart default
sudo virsh net-start default

sudo ip link set dev virbr0 up

sudo ip link set dev tap0 master virbr0


# use the  networking from above in the QEMU args

The actual bug is ebtalbes + dnsmasq doesn't work virsh networking. You can test it out:

sudo pacman -S libvirtd
virsh net-info --domain default
# works

Bugged:

sudo pacman -S ebtables dnsmasq

virsh net-info --domain default
# fails 

Fix:

sudo pacman -S iptables-nft

virsh net-info --domain default
# works

It's a strange bug, I though related to polkit, however, I found it incompatible with ebtables.

Possibly just with Arch, that the default network doesn't get installed.

Easy fix: replace iptables and ebtables with iptables-nft. If not, virsh hangs, sudo virsh hangs.

On the other hand, QEMU and Docker networking is really, really easy to debug. If bridged network is added, you'd have to start the bridge every time, I also think you need --net host

[Julioevm]

I was having issues with other machines communicating with a program running inside the macos docker, this might have something to do with it. If I get some free time ill try some of this stuff.

[sickcodes]

I was having issues with other machines communicating with a program running inside the macos docker, this might have something to do with it. If I get some free time ill try some of this stuff.

If you use the two -net command strings above in the xml file while using virt-manager you can switch between bridged and userMode networking. Haven't done bridged in the dockerfile but it wouldn't be much more than whats already here.

In summary:

Use virt-manager without docker Load the xml file from OSX-KVM

Try the networking commands above

Otherwise you can just add more ports to the QEMU command using the -e EXTRA env variable and also forwarding it again as a normal -p 111:2222 to the docker command.

[sickcodes]

Replaced iptables with iptables-nft

https://github.com/sickcodes/Docker-OSX/commit/8d6d0359b1fcebd368cf9d21e6955bacadb03526

[sickcodes]

~~e1000-82545em is preventing iMessage from working on Catalina and above~~

~~https://forums.unraid.net/topic/84288-catalina-network-bridging-weirdness/?do=findComment&comment=781612~~

~~However, a fix is available here: https://github.com/chris1111/AppleIntelE1000e~~

[mikob]

Couldn't get this working. Trying to have a port shared so that the host can access a server running on the mac on port 8080.

-e EXTRA='-netdev user,hostfwd=tcp::8080-:8080' This produces: qemu-system-x86_64: Parameter 'id' is missing

-e EXTRA='-netdev user,id=net0,hostfwd=tcp::8080-:8080' This produces: Duplicate ID 'net0' for netdev

-p 8080:8080 -e EXTRA='-netdev user,id=net1,hostfwd=tcp::8080-:8080' (qemu) qemu-system-x86_64: warning: netdev net1 has no peer let's the container run... but can't access the port on the host :/

[sickcodes]

I switched master to vmxnet3 in https://github.com/sickcodes/Docker-OSX/pull/160 as it did not affect iMessage and it massively increases speed.

[mikob]

@sickcodes Just unsure how to get port forwarding working. I suppose worst case scenario I can just forward via an ssh tunnel.

[sickcodes]

Maybe we can add another ${ADDITIONAL_PORTS} at the end of the line that has 5900 in it too for any other networking arguments for that interface

[sickcodes]

At the end of this line: https://github.com/sickcodes/Docker-OSX/blob/master/Dockerfile#L228

I'll add it in shortly

[sickcodes]

@mikob Added https://github.com/sickcodes/Docker-OSX/pull/162 let me test and then merge

[mikob]

@sickcodes just tested it out, it works! Thanks for your really speedy response!

[sickcodes]

@mikob No problem!

Here is a worked example for anyone else reading, as written here: https://github.com/sickcodes/Docker-OSX/pull/162#issuecomment-789490227

On the host

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e ADDITIONAL_PORTS='hostfwd=tcp::10023-:80,' \
    -p 10023:10023 \
    sickcodes/docker-osx:auto

Inside the container:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

brew install nginx
sudo sed -i -e 's/8080/80/' /usr/local/etc/nginx/nginx.conf
# sudo nginx -s stop
sudo nginx

nginx should appear on the host at port 10023

[mikob]

P.S. you don't need to install nginx for a test server, you can just do eg. python3 -m http.server 8080 to spin up a static test server that serves the CWD and it ships with python by default, which ships with osx by default.

[TimVanDyke]

I am unable to run:

sudo tee -a /etc/sysctl.conf <<< 'net.ipv4.ip_forward=1'
sudo sysctl -p
sudo pacman -S iptables-nft

sudo systemctl enable --now libvirtd.service
sudo systemctl enable --now virtlogd.service

sudo ip tuntap add dev tap0 mode tap
sudo ip link set tap0 up promisc on 

sudo virsh net-autostart default
sudo virsh net-start default

sudo ip link set dev virbr0 up

sudo ip link set dev tap0 master virbr0


# use the  networking from above in the QEMU args

without getting several errors. However, I believe this is the solution to the problems I'm having. Any help would be appreciated.

Output:

Package (1)        Old Version  New Version  Net Change

core/iptables-nft  1:1.8.9-1    1:1.8.9-1      0.00 MiB

Total Installed Size:  5.64 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] Y
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
:: Processing package changes...
reinstalling iptables-nft...
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
  Skipped: Current root is not booted.
(2/2) Arming ConditionNeedsUpdate...
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Authorization not available. Check if polkit service is running or see debug message for more information.
error: failed to connect to the hypervisor
error: Operation not supported: Cannot use direct socket mode if no URI is set

Authorization not available. Check if polkit service is running or see debug message for more information.
error: failed to connect to the hypervisor
error: Operation not supported: Cannot use direct socket mode if no URI is set

Cannot find device "virbr0"
Error: argument "virbr0" is wrong: Device does not exist

Here is my docker compose:

services:
    macos:
        container_name: 'MacOS'
        privileged: true
        network_mode: br0
        devices:
            - /dev/kvm
            - /dev/snd
            - /dev/null
        
        environment:
            - RAM=6
            - NETWORKING=vmxnet3
            - 'USERNAME=user'
            - 'PASSWORD=pass'
            - 'DISPLAY=${DISPLAY:-:0.0}'
        image: 'docker-osx-vnc:latest'
        
        #network_mode: host