frr
frr copied to clipboard
Published
20 hours ago •
FRRouting
FRRouting
EVPN-MH Split Horizon Filters Not Functional
Description
In my EVPN-MH setup, split horizon filters are not being implemented at the DF -- BUM traffic that is received on a shared segment at the non-DF will flood to the DF and be forwarded back down the shared segment.
Version
FRRouting 9.1 (R1) on Linux(6.6.16-amd64-vyos).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--enable-rpki' '--enable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'
How to reproduce
Configured topology derived in part from the EVPN-MH topotests.
Topology:
Network config
#Uplink
ip addr add 10.0.1.3/24 dev eth4
#Bridge
ip link add dev br9 type bridge stp_state 0
ip link set dev br9 type bridge vlan_filtering 1
ip link set dev br9 mtu 9216
ip link set dev br9 type bridge ageing_time 1800
ip link set dev br9 type bridge mcast_snooping 0
ip link set dev br9 type bridge vlan_stats_enabled 1
ip link set dev br9 up
bridge vlan add vid 999 dev br9 self
#ES Bond
ip link add dev bond9 type bond mode 802.3ad
ip link set dev bond9 type bond lacp_rate 1
ip link set dev bond9 type bond miimon 100
ip link set dev bond9 type bond xmit_hash_policy layer3+4
ip link set dev bond9 type bond min_links 1
ip link set dev bond9 type bond ad_actor_system fe:64:00:08:7b:00
ip link set dev eth2 down
ip link set dev eth2 master bond9
ip link set dev eth2 up
ip link set dev bond9 up
ip link set dev bond9 master br9
bridge link set dev bond9 priority 8
bridge vlan del vid 1 dev bond9
bridge vlan del vid 1 untagged pvid dev bond9
bridge vlan add vid 999 dev bond9
bridge vlan add vid 999 untagged pvid dev bond9
#VXLAN
ip link add dev vx-999 type vxlan id 999 dstport 4789
ip link set dev vx-999 type vxlan nolearning
ip link set dev vx-999 type vxlan local 10.0.1.3
ip link set dev vx-999 type vxlan ttl 64
ip link set dev vx-999 mtu 9152
ip link set dev vx-999 up
ip link set dev vx-999 master br9
bridge link set dev vx-999 neigh_suppress on
bridge link set dev vx-999 learning off
bridge link set dev vx-999 priority 8
bridge vlan del vid 1 dev vx-999
bridge vlan del vid 1 untagged pvid dev vx-999
bridge vlan add vid 999 dev vx-999
bridge vlan add vid 999 untagged pvid dev vx-999
#SVI
ip link add link br9 name vlan999 type vlan id 999 protocol 802.1q
ip addr add 10.99.0.3/24 dev vlan999
ip link set dev vlan999 up
ip link add link vlan999 name vlan999-v0 type macvlan mode private
ip link set dev vlan999-v0 address 00:00:5e:00:01:99
ip link set dev vlan999-v0 up
ip addr add 10.99.0.1/24 dev vlan999-v0
FRR Config:
frr version 9.1
frr defaults traditional
hostname R1
no ipv6 forwarding
service integrated-vtysh-config
!
interface bond9
evpn mh es-df-pref 20000
evpn mh es-id 9
evpn mh es-sys-mac fe:64:00:08:7b:00
exit
!
interface eth4
evpn mh uplink
exit
!
router bgp 65000
bgp router-id 10.0.0.3
bgp log-neighbor-changes
no bgp ebgp-requires-policy
no bgp default ipv4-unicast
no bgp network import-check
timers bgp 10 30
neighbor 10.0.1.4 remote-as 65000
neighbor 10.0.1.4 update-source 10.0.1.3
!
address-family l2vpn evpn
neighbor 10.0.1.4 activate
neighbor 10.0.1.4 soft-reconfiguration inbound
advertise-all-vni
advertise-svi-ip
exit-address-family
exit
Network state:
vyos@R1:/$ bridge -d link
34: bond9: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 master br9 state forwarding priority 8 cost 4
hairpin off guard off root_block off fastleave off learning on flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress off neigh_vlan_suppress off vlan_tunnel off isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
35: vx-999: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9152 master br9 state forwarding priority 8 cost 100
hairpin off guard off root_block off fastleave off learning off flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress on neigh_vlan_suppress off vlan_tunnel off isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
vyos@R1:/$ bridge -d vlan
port vlan-id
br9 1 PVID Egress Untagged
state forwarding mcast_router 1 neigh_suppress off
999
state forwarding mcast_router 1 neigh_suppress off
bond9 999 PVID Egress Untagged
state forwarding mcast_router 1 neigh_suppress off
vx-999 999 PVID Egress Untagged
state forwarding mcast_router 1 neigh_suppress off
vyos@R1:/$ ip -d link
3: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond9 state UP mode DEFAULT group default qlen 1000
link/ether ea:55:80:c6:ed:81 brd ff:ff:ff:ff:ff:ff permaddr 00:f0:cb:fe:c3:a3 promiscuity 1 allmulti 1 minmtu 68 maxmtu 9216
bond_slave state ACTIVE mii_status UP link_failure_count 3 perm_hwaddr 00:f0:cb:fe:c3:a3 queue_id 0 prio 0 ad_aggregator_id 1 ad_actor_oper_port_state 79 ad_actor_oper_port_state_str <active,short_timeout,aggregating,in_sync,defaulted> ad_partner_oper_port_state 1 ad_partner_oper_port_state_str <active> numtxqueues 4 numrxqueues 4 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536 parentbus pci parentdev 0000:02:00.0
altname enp2s0
4: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9152 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:f0:cb:fe:c0:a7 brd ff:ff:ff:ff:ff:ff promiscuity 0 allmulti 0 minmtu 68 maxmtu 9710 numtxqueues 71 numrxqueues 71 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536 parentbus pci parentdev 0000:04:00.0
altname enp4s0f0
33: br9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 8a:a0:59:68:91:f5 brd ff:ff:ff:ff:ff:ff promiscuity 1 allmulti 0 minmtu 68 maxmtu 65535
bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 1800 stp_state 0 priority 32768 vlan_filtering 1 vlan_protocol 802.1Q bridge_id 8000.8a:a0:59:68:91:f5 designated_root 8000.8a:a0:59:68:91:f5 root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer 0.00 tcn_timer 0.00 topology_change_timer 0.00 gc_timer 6.36 vlan_default_pvid 1 vlan_stats_enabled 1 vlan_stats_per_port 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 0 no_linklocal_learn 0 mcast_vlan_snooping 0 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 16 mcast_hash_max 4096 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3125 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
34: bond9: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master br9 state UP mode DEFAULT group default qlen 1000
link/ether ea:55:80:c6:ed:81 brd ff:ff:ff:ff:ff:ff promiscuity 1 allmulti 1 minmtu 68 maxmtu 65535
bond mode 802.3ad miimon 100 updelay 0 downdelay 0 peer_notify_delay 0 use_carrier 1 arp_interval 0 arp_missed_max 2 arp_validate none arp_all_targets any primary_reselect always fail_over_mac none xmit_hash_policy layer3+4 resend_igmp 1 num_grat_arp 1 all_slaves_active 0 min_links 1 lp_interval 1 packets_per_slave 1 lacp_active on lacp_rate fast ad_select stable ad_aggregator 1 ad_num_ports 1 ad_actor_key 11 ad_partner_key 1 ad_partner_mac 00:00:00:00:00:00 tlb_dynamic_lb 1
bridge_slave state forwarding priority 8 cost 4 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x2001 port_no 0x1 designated_port 8193 designated_cost 0 designated_bridge 8000.8a:a0:59:68:91:f5 designated_root 8000.8a:a0:59:68:91:f5 hold_timer 0.00 message_age_timer 0.00 forward_delay_timer 0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on bcast_flood on mcast_to_unicast off neigh_suppress off neigh_vlan_suppress off group_fwd_mask 0 group_fwd_mask_str 0x0 vlan_tunnel off isolated off locked off mab off numtxqueues 16 numrxqueues 16 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
35: vx-999: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9152 qdisc noqueue master br9 state UNKNOWN mode DEFAULT group default qlen 1000
link/ether a6:73:20:51:91:c0 brd ff:ff:ff:ff:ff:ff promiscuity 1 allmulti 1 minmtu 68 maxmtu 65535
vxlan id 999 local 10.0.1.3 srcport 0 0 dstport 4789 ttl 64 ageing 300 nolearning
bridge_slave state forwarding priority 8 cost 100 hairpin off guard off root_block off fastleave off learning off flood on port_id 0x2002 port_no 0x2 designated_port 8194 designated_cost 0 designated_bridge 8000.8a:a0:59:68:91:f5 designated_root 8000.8a:a0:59:68:91:f5 hold_timer 0.00 message_age_timer 0.00 forward_delay_timer 0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on bcast_flood on mcast_to_unicast off neigh_suppress on neigh_vlan_suppress off group_fwd_mask 0 group_fwd_mask_str 0x0 vlan_tunnel off isolated off locked off mab off numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
36: vlan999@br9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 8a:a0:59:68:91:f5 brd ff:ff:ff:ff:ff:ff promiscuity 0 allmulti 0 minmtu 0 maxmtu 65535
vlan protocol 802.1Q id 999 <REORDER_HDR> numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
37: vlan999-v0@vlan999: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 00:00:5e:00:01:99 brd ff:ff:ff:ff:ff:ff promiscuity 0 allmulti 0 minmtu 68 maxmtu 65535
macvlan mode private bcqueuelen 1000 usedbcqueuelen 1000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
R1# show evpn es detail
ESI: 03:fe:64:00:08:7b:00:00:00:09
Type: Local,Remote
Interface: bond9
State: up
Bridge port: yes
Ready for BGP: yes
VNI Count: 1
MAC Count: 12
DF status: df
DF preference: 20000
Nexthop group: 536870913
VTEPs:
10.0.1.4 df_alg: preference df_pref: 10000 nh: 268435458
R1# show evpn vni 999
VNI: 999
Type: L2
Vlan: 999
Bridge: br9
Tenant VRF: default
VxLAN interface: vx-999
VxLAN ifIndex: 35
SVI interface: vlan999
SVI ifIndex: 36
Local VTEP IP: 10.0.1.3
Mcast group: 0.0.0.0
Remote VTEPs for this VNI:
10.0.1.4 flood: HER
Number of MACs (local and remote) known for this VNI: 13
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 4
Advertise-gw-macip: No
Advertise-svi-macip: No
BUM Traffic received at R2 (non-DF) is flooded to R1 (DF), and then forwarded back down the shared segment:
vyos@R2:/$ tcpdump -ennli eth2 inbound
14:08:27.179443 d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, ethertype Realtek protocols (0x8899), length 60: d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, Realtek unknown type 0x25
14:08:27.975596 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 344: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 302
14:08:27.983774 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 386: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 344
vyos@R1:/$ tcpdump -ennli eth2 outbound
14:08:26.876979 d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, ethertype Realtek protocols (0x8899), length 60: d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, Realtek unknown type 0x25
14:08:27.673456 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 344: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 302
14:08:27.681135 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 386: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 344
Expected behavior
Split-horizon filters should prevent traffic flooded from the non-DF VTEP on a shared segment from being forwarded back down the shared segment at the DF VTEP. The codebase indicates the SPH filters are implemented in the data-plane, but it is unclear to me how exactly they are implemented. Are the filters supposed to work with the kernel as the data-plane?
Actual behavior
BUM Traffic received at R2 (non-DF) is flooded to R1 (DF), and then forwarded back down the shared segment:
vyos@R2:/$ tcpdump -ennli eth2 inbound
14:08:27.179443 d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, ethertype Realtek protocols (0x8899), length 60: d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, Realtek unknown type 0x25
14:08:27.975596 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 344: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 302
14:08:27.983774 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 386: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 344
vyos@R1:/$ tcpdump -ennli eth2 outbound
14:08:26.876979 d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, ethertype Realtek protocols (0x8899), length 60: d8:ec:e5:9d:17:a0 > ff:ff:ff:ff:ff:ff, Realtek unknown type 0x25
14:08:27.673456 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 344: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 302
14:08:27.681135 14:cb:19:78:8c:80 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 386: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 14:cb:19:78:8c:80, length 344
Additional context
Seems related to discussion towards end of https://github.com/FRRouting/frr/discussions/11487
Checklist
- [X] I have searched the open issues for this bug.
- [X] I have not included sensitive information in this report.
Can anyone address this? If EVPN-MH support in FRR is incomplete, that is understandable. What is not understandable is FRR pretending it is complete in the documentation. The documentation should ideally be prefaced that EVPN-MH in FRR is only partially complete, and that cumulus or another system is necessary for full functionality. But at the very minimum the following paragraphs should removed from the documentation, given that the statements are completely false.
"BUM traffic is rxed via the overlay by all PEs attached to a server but only the DF can forward the de-capsulated traffic to the access port. To accommodate that non-DF filters are installed in the dataplane to drop the traffic.
Similarly traffic received from ES peers via the overlay cannot be forwarded to the server. This is split-horizon-filtering with local bias."
This issue is stale because it has been open 180 days with no activity. Comment or remove the autoclose label in order to avoid having this issue closed.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue will be automatically closed in the specified period unless there is further activity.
![frrbot[bot] avatar](https://avatars.githubusercontent.com/in/136408?v=4 "Published by a pub.dev verified publisher"){kind=small}