prevent default fallback route lookup from user-defined VRF table to local table(default vrf).

[sumanbrcm]

What I did By default fallback to local table if l3mdev table lookup fails is enabled in kernel. This enables packet to move to default-vrf if route lookup in non-default-vrf fails. To disable this fallback feature below IPv4 & IPv6 rules are added to FIB Routing Policy Data Base.

ip ru add pref 1001 l3mdev unreachable ip -6 ru add pref 1001 l3mdev unreachable

Why I did it This fix is needed as it prevents default fallback route lookup from user-defined VRF table to local table(default vrf).

How I verified it Please refer to below output after fix : admin@sonic: ip -4 rule ls 1000: from all lookup [l3mdev-table] 1003: from 10.59.133.11 lookup mgmt 1004: from all to 10.0.0.0/8 lookup mgmt 32765: from all lookup local 32766: from all lookup main 32767: from all lookup default admin@sonic:

admin@sonic: ip -6 rule ls 1000: from all lookup [l3mdev-table] 1003: from 2100::2 lookup mgmt 32765: from all lookup local 32766: from all lookup main admin@sonic:

[preetham-singh]

Adding few details for adding l3mdev unreachable rule:

If l3mdev unreachable rule is not present, route lookup by default falls back to local table(default vrf routing table which has connected prefixes). If connected prefix is present in local table for which lookup is performed in non-default vrf, result will lead to nexthop in local table causing packet leak from non-default vrf to default vrf. Below are some linux output for route lookup before and after adding l3mdev unreachable rule.

Before Fix:

root@sonic:/home/admin# show vrf VRF Interfaces

---

Vrf-red Ethernet56

root@sonic:/home/admin# ip -4 ru ls 1000: from all lookup [l3mdev-table] 1001: from all lookup local 32766: from all lookup main 32767: from all lookup default

root@sonic:/home/admin# ip -4 route get 24.0.0.100 vrf Vrf-red 24.0.0.100 dev Ethernet56 table 1001 src 24.0.0.2 uid 0 cache

root@sonic:/home/admin# config interface ip remove Ethernet56 24.0.0.2/24

root@sonic:/home/admin# ip -4 route get 24.0.0.100 vrf Vrf-red 24.0.0.100 dev Ethernet24 src 24.0.0.2 uid 0 cache

root@sonic:/home/admin# ip -4 route get 24.0.0.100 24.0.0.100 dev Ethernet24 src 24.0.0.2 uid 0 cache

root@sonic:/home/admin# ip -s addr show Ethernet24 36: Ethernet24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast state UP group default qlen 1000 link/ether 3c:2c:99:2d:84:35 brd ff:ff:ff:ff:ff:ff inet 24.0.0.2/24 brd 24.0.0.255 scope global Ethernet24 valid_lft forever preferred_lft forever inet6 fe80::3e2c:99ff:fe2d:8435/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped overrun mcast 14396 62 0 2 0 0 TX: bytes packets errors dropped carrier collsns 14572 63 0 0 0 0

root@sonic:/home/admin# ip -s addr show Ethernet56 44: Ethernet56: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000 link/ether 3c:2c:99:2d:84:35 brd ff:ff:ff:ff:ff:ff inet 56.0.0.2/24 brd 56.0.0.255 scope global Ethernet56 valid_lft forever preferred_lft forever inet6 fe80::3e2c:99ff:fe2d:8435/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped overrun mcast 218601 1633 0 5 0 0 TX: bytes packets errors dropped carrier collsns 16973 98 0 0 0 0 root@sonic:/home/admin#

After fix:

root@sonic:~# ip -4 ru ls 1000: from all lookup [l3mdev-table] 1001: from all lookup [l3mdev-table] unreachable 1002: from all lookup local 32766: from all lookup main 32767: from all lookup default

root@sonic:~# show vrf VRF Interfaces

---

Vrf-red Ethernet48

root@sonic:~# ip -s addr show Ethernet48 27: Ethernet48: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000 link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff inet 24.0.0.1/24 brd 24.0.0.255 scope global Ethernet48 valid_lft forever preferred_lft forever inet 48.0.0.1/24 brd 48.0.0.255 scope global Ethernet48 valid_lft forever preferred_lft forever inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped overrun mcast 16018 69 0 0 0 0 TX: bytes packets errors dropped carrier collsns 16657 73 0 0 0 0

root@sonic:~# ip -s addr show Ethernet24 25: Ethernet24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast state UP group default qlen 1000 link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff inet 24.0.0.1/24 brd 24.0.0.255 scope global Ethernet24 valid_lft forever preferred_lft forever inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped overrun mcast 18322 105 0 0 0 0 TX: bytes packets errors dropped carrier collsns 15719 67 0 0 0 0

root@sonic:~# ip -4 route get 24.0.0.100 vrf Vrf-red 24.0.0.100 dev Ethernet48 table 1001 src 24.0.0.1 uid 0 cache

root@sonic:~# config interface ip remove Ethernet48 24.0.0.1/24

root@sonic:~# ip -s addr show Ethernet48 27: Ethernet48: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000 link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff inet 48.0.0.1/24 brd 48.0.0.255 scope global Ethernet48 valid_lft forever preferred_lft forever inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped overrun mcast 16259 70 0 0 0 0 TX: bytes packets errors dropped carrier collsns 16899 74 0 0 0 0

root@sonic:~# ip -4 route get 24.0.0.100 vrf Vrf-red RTNETLINK answers: Network is unreachable

root@sonic:~#

[ben-gale]

@lguohan - can we get some review on this please?