libnetwork
libnetwork copied to clipboard
moby
IP6Tables breaks IPv6 ICC for internal bridge networks
IPv6 connectivity between containers (ICC) in internal bridge networks seems broken. I've enabled IPv6 and IP6Tables for the daemon since I want basic firewalling and isolation between Docker IPv6 networks (not because I need NAT66). I believe the broken IPv6 ICC is caused by the current IP6Tables rules only allowing forwarding between addresses within the defined IPv6 subnet and not any multicast addresses, which IPv6 neighbor discovery depends on (IPv6 NDP is L3 unlike IPv4 ARP which is L2). For non-internal IPv6 bridge networks, IPv6 ICC works fine.
If my assumption regarding why it's broken is correct, I suppose the two alternatives of solving this is to either reorder the IP6Tables rules such that traffic within the bridge interface is always allowed regardless of the IPv6 addresses matching the subnet, or explicitly allowing NDP-specific ICMPv6 types within the bridge interface.
Expected behavior
Have IPv6 and IPv6Tables enabled in the daemon config. Create an IPv6-enabled bridge network and start to containers connected to it. They should be able to ping eachother using both IPv4 and IPv6.
Actual behavior
The two containers are able to ping eachother using IPv4 only. They can however ping the gateway (which still exists for internal networks for some reason) and vice versa (which I assume counts as "input" and "output" instead of "forward"). This renders internal IPv6 bridge networks completely useless.
Steps to reproduce the behavior
I've created two IPv6-enabled bridge networks: one "non-internal"/normal and one internal.
Non-internal (created as br-5ebc8c4b02b2): docker network create --subnet=10.0.100.0/24 --ipv6 --subnet=fd69:2256:f372:1dd2::/64 test-noninternal
Internal (created as br-d6680826d665): docker network create --subnet=172.31.0.0/24 --ipv6 --subnet=fdb7:cb86:7dcd:dab0::/64 --internal test-internal
Created IP6Tables (ip6tables -L -vn), (including some non-Docker rules):
Chain INPUT (policy DROP 119K packets, 9149K bytes)
pkts bytes target prot opt in out source destination
7671 4219K ACCEPT all * * ::/0 ::/0 ctstate RELATED,ESTABLISHED
6 434 DROP all * * ::/0 ::/0 ctstate INVALID
4 313 ACCEPT all lo * ::/0 ::/0
24 2112 ACCEPT icmpv6 * * ::/0 ::/0 ipv6-icmptype 128
52722 2949K ACCEPT icmpv6 * * ::/0 ::/0 ipv6-icmptype 133
1 176 ACCEPT icmpv6 * * ::/0 ::/0 ipv6-icmptype 134
2221K 160M ACCEPT icmpv6 * * ::/0 ::/0 ipv6-icmptype 135
6043 435K ACCEPT icmpv6 * * ::/0 ::/0 ipv6-icmptype 136
0 0 ACCEPT udp * * ::/0 ::/0 udp dpt:546
0 0 ACCEPT udp * * ::/0 ::/0 udp dpt:547
8 532 ACCEPT tcp * * ::/0 ::/0 tcp dpt:22
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
766 117K DOCKER-ISOLATION-STAGE-1 all * * ::/0 ::/0
3 312 ACCEPT all * br-5ebc8c4b02b2 ::/0 ::/0 ctstate RELATED,ESTABLISHED
13 1024 DOCKER all * br-5ebc8c4b02b2 ::/0 ::/0
0 0 ACCEPT all br-5ebc8c4b02b2 !br-5ebc8c4b02b2 ::/0 ::/0
13 1024 ACCEPT all br-5ebc8c4b02b2 br-5ebc8c4b02b2 ::/0 ::/0
0 0 ACCEPT all br-d6680826d665 br-d6680826d665 ::/0 ::/0
0 0 ACCEPT all * docker0 ::/0 ::/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all * docker0 ::/0 ::/0
0 0 ACCEPT all docker0 !docker0 ::/0 ::/0
0 0 ACCEPT all docker0 docker0 ::/0 ::/0
36M 16G ACCEPT all br0 br0 ::/0 ::/0
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
298K 27M ACCEPT all * * ::/0 ::/0
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all br-5ebc8c4b02b2 !br-5ebc8c4b02b2 ::/0 ::/0
98 6992 DROP all * br-d6680826d665 !fdb7:cb86:7dcd:dab0::/64 ::/0
3054 220K DROP all br-d6680826d665 * ::/0 !fdb7:cb86:7dcd:dab0::/64
0 0 DOCKER-ISOLATION-STAGE-2 all docker0 !docker0 ::/0 ::/0
32M 15G RETURN all * * ::/0 ::/0
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-2 (2 references)
pkts bytes target prot opt in out source destination
0 0 DROP all * br-5ebc8c4b02b2 ::/0 ::/0
0 0 DROP all * docker0 ::/0 ::/0
0 0 RETURN all * * ::/0 ::/0
Note that multicast addresses (which NDP depends on) ends up getting dropped in DOCKER-ISOLATION-STAGE-1 for the internal net (br-d6680826d665), but not for the noninternal net (br-5ebc8c4b02b2).
Docker info and version
See gist.
Additional environment details (AWS, VirtualBox, physical, etc.)
daemon.json:
{
"dns": ["1.1.1.1", "2606:4700:4700::1111"],
"experimental": true,
"metrics-addr": "[::]:9323",
"ipv6": true,
"ip6tables": true,
"fixed-cidr-v6": "fd18:bddb:1b0d:6f7c::/64"
}
Ran into this today and can confirm @HON95 conclusion. The ip6tables rules are broken and do not take into account IPv6 neighbor discovery.
This was fixed in moby/moby#45649 and has been backported to v24.0 in moby/moby#46214.
