kernelkit
IPv6 auto-conf for bridge interfaces gets random link identifier instead of MAC based (eui64)
Reproduced on latest infix (2024-03-25) but seems to have been like this all the time.
When using IPv6 autoconfiguration on a bridge interface, the fe80:: address will differ for each boot. The link identifier seems to be random rather than based on the bridge MAC (which is inherited from the first/lowest port attached)
admin@infix-06-11-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ipv4 169.254.4.73/16 (random)
│ ipv4 198.18.101.168/24 (dhcp)
│ ipv6 fe80::206a:30ff:fe2d:5db4/64 (link-layer)
├ x1 bridge LOWER-LAYER-DOWN
├ x2 bridge LOWER-LAYER-DOWN
...
Next boot
admin@infix-06-11-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ipv4 169.254.4.73/16 (random)
│ ipv4 198.18.101.168/24 (dhcp)
│ ipv6 fe80::9cc2:1eff:fea5:9437/64 (link-layer)
├ x1 bridge LOWER-LAYER-DOWN
├ x2 bridge LOWER-LAYER-DOWN
├ x3 bridge LOWER-LAYER-DOWN
├ x4 bridge LOWER-LAYER-DOWN
...
Note that the address claims to be based on MAC address (type
link-layerrather thanrandomin listing above).
admin@infix-06-11-00:~$ ip -d -j link show dev br0 | jq -r '.[].inet6_addr_gen_mode'
eui64
admin@infix-06-11-00:~$
Cannot reproduce (on my Qemu system). Could you attach the startup-config you used to trigger this (censoring any customer specifics)?
Also, did this trigger from power-on (using startup-config) or did it happen at runtime, i.e., reconfiguring the device to use bridge + ipv6? Because in the latter case we've seen issues with enabling settings post-boot that don't "take" until after a save and reboot.
Addendum: I was a bit curious if we had missed something else, and I found another sysctl setting called use_tempaddr. Also some interesting comments here. We don't do anything currently with this setting and it seems to be disabled (0) by default.
J-O posted a follow-up in a side channel. What follows is my translation from Swedish, any errors are mine.
"I've tested in GNS3 with an image from April 6. That system shows the same behavior as reported initially, i.e., random IPv6 address despite address mode reported to be
eui64."
admin@infix-ae-00-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ethernet UP 0c:9b:6a:ae:00:00
│ ipv6 fe80::a401:f4ff:fe7c:5b02/64 (link-layer)
└ eth0 bridge FORWARDING
eth1 ethernet DOWN 0c:9b:6a:ae:00:01
eth2 ethernet DOWN 0c:9b:6a:ae:00:02
eth3 ethernet DOWN 0c:9b:6a:ae:00:03
eth4 ethernet DOWN 0c:9b:6a:ae:00:04
eth5 ethernet DOWN 0c:9b:6a:ae:00:05
eth6 ethernet DOWN 0c:9b:6a:ae:00:06
eth7 ethernet DOWN 0c:9b:6a:ae:00:07
eth8 ethernet DOWN 0c:9b:6a:ae:00:08
eth9 ethernet DOWN 0c:9b:6a:ae:00:09
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
admin@infix-ae-00-00:/>
reboot
admin@infix-ae-00-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ethernet UP 0c:9b:6a:ae:00:00
│ ipv6 fe80::4c2:aaff:fe55:65ae/64 (link-layer)
└ eth0 bridge FORWARDING
eth1 ethernet DOWN 0c:9b:6a:ae:00:01
eth2 ethernet DOWN 0c:9b:6a:ae:00:02
eth3 ethernet DOWN 0c:9b:6a:ae:00:03
eth4 ethernet DOWN 0c:9b:6a:ae:00:04
eth5 ethernet DOWN 0c:9b:6a:ae:00:05
eth6 ethernet DOWN 0c:9b:6a:ae:00:06
eth7 ethernet DOWN 0c:9b:6a:ae:00:07
eth8 ethernet DOWN 0c:9b:6a:ae:00:08
eth9 ethernet DOWN 0c:9b:6a:ae:00:09
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
admin@infix-ae-00-00:/>
admin@infix-ae-00-00:/> show version
Infix -- a Network Operating System 320410b -- Apr 6 11:39 UTC 2024
admin@infix-ae-00-00:/>
admin@infix-ae-00-00:/> exit
admin@infix-ae-00-00:~$ ip -d -j link show dev br0 | jq -r '.[].inet6_addr_gen_mode'
eui64
admin@infix-ae-00-00:~$
An example with qemu using infix latest as of April 16 (built April 15 as shown below) Configuration included, but in short.
- Default config
- Create br0, enable IPv6
- Let e0 be a bridge-port of br0
- That's it
Startup Config attached. startup-config.txt
PC~:/tmp/infix-x86_64 $ ./qemu.sh
Starting Qemu :: Ctrl-a x -- exit | Ctrl-a c -- toggle console/monitor
● ● ● Infix -- a Network Operating System e06bcb6 ═════════════════════
[ OK ] Mounting filesystems from /etc/fstab
[ OK ] Restoring system clock (UTC) from RTC
[ OK ] Seeding random number generator
...
admin@infix-00-00-00:~$ cli
See the 'help' command for an introduction to the system
admin@infix-00-00-00:/> show version
Infix -- a Network Operating System e06bcb6 -- Apr 15 15:11 UTC 2024
admin@infix-00-00-00:/>
Configuration:
admin@infix-00-00-00:/> configure
admin@infix-00-00-00:/config/> edit interface br0
admin@infix-00-00-00:/config/interface/br0/> set ipv6 enabled
admin@infix-00-00-00:/config/interface/br0/> show
type bridge;
ipv6 {
enabled true;
}
admin@infix-00-00-00:/config/interface/br0/> end
admin@infix-00-00-00:/config/> set interface e0 bridge-port bridge br0
admin@infix-00-00-00:/config/> leave
admin@infix-00-00-00:/> copy running-config startup-config
admin@infix-00-00-00:/>
Show IP assignment
admin@infix-00-00-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ethernet UP 02:00:00:00:00:00
│ ipv6 fe80::dc7c:aeff:fef4:226/64 (link-layer)
└ e0 bridge FORWARDING
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
admin@infix-00-00-00:/>
reboot and login again IPv6 changed, despite stating "link-layer" (not random)
admin@infix-00-00-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ethernet UP 02:00:00:00:00:00
│ ipv6 fe80::4c11:46ff:fee5:b3de/64 (link-layer)
└ e0 bridge FORWARDING
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
admin@infix-00-00-00:/>
reboot and login again IPv6 changed, despite stating "link-layer" (not random)
admin@infix-00-00-00:/> show interfaces
INTERFACE PROTOCOL STATE DATA
br0 bridge
│ ethernet UP 02:00:00:00:00:00
│ ipv6 fe80::acb2:39ff:fe03:6d1e/64 (link-layer)
└ e0 bridge FORWARDING
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
admin@infix-00-00-00:/>
In later infix versions, IPv6 address gen mode is no longer shown (null rather than eui64)
admin@infix-00-00-00:/> exit
admin@infix-00-00-00:~$ ip -d -j link show dev br0 | jq -r '.[].inet6_addr_gen_mode'
null
admin@infix-00-00-00:~$
Full output
admin@infix-00-00-00:~$ ip -d -j link show dev br0 | jq -r
[
{
"ifindex": 3,
"ifname": "br0",
"flags": [
"BROADCAST",
"MULTICAST",
"UP",
"LOWER_UP"
],
"mtu": 1500,
"qdisc": "noqueue",
"operstate": "UP",
"linkmode": "DEFAULT",
"group": "default",
"txqlen": 1000,
"link_type": "ether",
"address": "02:00:00:00:00:00",
"broadcast": "ff:ff:ff:ff:ff:ff",
"promiscuity": 0,
"allmulti": 0,
"min_mtu": 68,
"max_mtu": 65535,
"linkinfo": {
"info_kind": "bridge",
"info_data": {
"forward_delay": 1500,
"hello_time": 200,
"max_age": 2000,
"ageing_time": 30000,
"stp_state": 0,
"priority": 32768,
"vlan_filtering": 0,
"vlan_protocol": "802.1Q",
"bridge_id": "8000.2:0:0:0:0:0",
"root_id": "8000.2:0:0:0:0:0",
"root_port": 0,
"root_path_cost": 0,
"topology_change": 0,
"topology_change_detected": 0,
"hello_timer": 0.00,
"tcn_timer": 0.00,
"topology_change_timer": 0.00,
"gc_timer": 216.47,
"vlan_default_pvid": 0,
"vlan_stats_enabled": 0,
"vlan_stats_per_port": 0,
"group_fwd_mask": "0",
"group_addr": "01:80:c2:00:00:00",
"mcast_snooping": 1,
"no_linklocal_learn": 0,
"mcast_vlan_snooping": 0,
"mst_enabled": 0,
"mcast_flood_always": 1,
"mcast_router": 1,
"mcast_query_use_ifaddr": 0,
"mcast_querier": 0,
"mcast_hash_elasticity": 16,
"mcast_hash_max": 4096,
"mcast_last_member_cnt": 2,
"mcast_startup_query_cnt": 2,
"mcast_last_member_intvl": 100,
"mcast_membership_intvl": 26000,
"mcast_querier_intvl": 25500,
"mcast_query_intvl": 12500,
"mcast_query_response_intvl": 1000,
"mcast_startup_query_intvl": 3124,
"mcast_stats_enabled": 0,
"mcast_igmp_version": 2,
"mcast_mld_version": 1,
"nf_call_iptables": 0,
"nf_call_ip6tables": 0,
"nf_call_arptables": 0
}
},
"num_tx_queues": 1,
"num_rx_queues": 1,
"gso_max_size": 65536,
"gso_max_segs": 65535,
"tso_max_size": 65536,
"tso_max_segs": 65535,
"gro_max_size": 65536,
"gso_ipv4_max_size": 65536,
"gro_ipv4_max_size": 65536
}
]
admin@infix-00-00-00:~$
Reproduced. Root cause is the initial random MAC address assigned to the bridge before we add any bridge ports. The fix is to ensure the bridge is always created with the system base mac.
For Qemu/Qeneth:
- Update qeneth template (dut1:eth0 -> 02:00:00:00:01:01)
- probe -> extract base mac or take eth0 MAC - 1
- confd gen bridge: if custom -> set custom, else set base mac, else random
Re: IPv6 addrgen mode no longer shown, apparently the iproute2 upgrade caused this. In v6.5 they added support for "filtering stats", a kernel feature that has been available since Linux 2015, behind which the IPv6 addrgen mode is hidden.
From now on, use:
ip -d -j -s link show dev br0 | jq -r '.[].inet6_addr_gen_mode'
The -s is for statistics.
The fix to this has been moved to a separate branch, https://github.com/kernelkit/infix/tree/stable-bridge-mac, because it for some reason break IGMP functionality. Pending investigation.
I had a quick look at why the test-suite fails with the fix for this in https://github.com/kernelkit/infix/tree/stable-bridge-mac applied.
The test igmp-basic.py fails with the message not ok 6 - Verify that the group is no longer forwarded to the non-member. In this scenario the test joins a multicast group on interface X and attempts to ensure traffic is now longer flooded to the non-member interface Y. After the mentioned commit is applied nojoin_ns.must_not_receive("ip dst 224.1.1.1") fails, indicating that tcpdump saw multicast traffic on the non-member interface.
I manually verified that bridge snooping still works and that traffic isn't flooded as the test failure suggests. This points to either a race condition in the test or some strange first-time behavior when joining a group in a pristine system. Investigation will continue.
Discussed AFK with @wkz about your findings from yesterday, and I think the best way forward here is kernelkit/mcd#5. I.e., fix in userspace.