ops icon indicating copy to clipboard operation
ops copied to clipboard

Fix net setup. Right now it drops the connection

Open tijoytom-zz opened this issue 5 years ago • 3 comments

setting up the bridge network drops the net connection, figure out a way to do it otherwise. Also the dhcp request for bridge not always succeed.

tijoytom-zz avatar Jan 25 '19 00:01 tijoytom-zz

Xen(d) used to attempt direct bridge networking for guest and found it flaky (not to mention the wifi problem):

Prior to Xen 4.1 when xend started up it would run the network-bridge script which would reconfigure any existing physical network configuration into a bridged network configuration i.e. it would create a bridge, move the IP address from the physical device to the bridge, add the physical device to the bridge etc. However this was fragile and prone to breaking and therefore is no longer recommended.

Docker and Libvirt use an isolated bridge and then use Nat + iptables forwarding to setup communication with the outside for guests.

When the libvirt default network is running, you will see an isolated bridge device. This device explicitly does NOT have any physical interfaces added, since it uses NAT + forwarding to connect to outside world. Do not add interfaces

When you create or remove a user-defined bridge or connect or disconnect a container from a user-defined bridge, Docker uses tools specific to the operating system to manage the underlying network infrastructure (such as adding or removing bridge devices or configuring iptables rules on Linux). These details should be considered implementation details. Let Docker manage your user-defined networks for you.

Should ops follow suite here? Using Nat + routing by default could be best. A parameter to accept a pre-configured bridge or tap device (qemu style) could act as an alternative to this default to add more flexibility.

bridge name	bridge id		STP enabled	interfaces
docker0		8000.02427437ad99	no		vethe4344e2

I have a debian domU running with docker installed running a centos image. As you can see above Docker attaches one end of the veth pair to the bridge and sets up routing rules for outside commo.

  1. https://wiki.xenproject.org/wiki/Xen_Networking
  2. https://wiki.libvirt.org/page/Networking
  3. https://docs.docker.com/network/bridge/

ExternalReality avatar Jan 25 '19 05:01 ExternalReality

I haven't looked in to the details ye, but Nat + IP forwarding do not perform as well as bridged right?

tijoytom-zz avatar Jan 25 '19 07:01 tijoytom-zz

some considerations here (most of this is really clarifying documentation) - i'm not aware of any actual problem (when is dhcp not responding?):

  • your avg. cloud vm w/one interface is not going to be a great usecase outside of demoware cause of the perf tax - these aren't containers (we are very close to having native gce/aws deploys as vms not on top - which pushes this responsibility onto the respective providers and imo completely nixes this usecase to begin with)

  • nested virt ala google is a usecase - although I haven't touched it i'd be very surprised if you can't attach 2 interfaces to it - if that's the case that should be documented

  • most bare metal servers (heztner/packet/bring your own) I'm familiar with always have more in than one interface - this obviates the connection drop as you can use one interface for a control plane (ssh/etc.) and one for a data plane - once again document

  • ip-forwarding is going to always be the case for wireless (osx) setups even when a bridge exists - github.com/deferpanic/virgo was setup and implied for this (wireless ap rejects the frames)

  • every real setup has a certain implication that there is going to be at least one - prob in many cases more priv. networks

eyberg avatar Jan 25 '19 14:01 eyberg