frr icon indicating copy to clipboard operation
frr copied to clipboard

Published 20 hours ago • verified publisher icon FRRouting

staticd: route not able add into kernel

Open punith-shivakumar opened this issue 1 year ago • 10 comments

staticd routes does not show up in kernel table. Issue seen from second route with self interface as gateway.

  • [x] Did you check if this is a duplicate issue?
  • [x] Did you test it on the latest FRRouting/frr master branch?

To Reproduce 1. Interface config used

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue group 0x04 nfmark 0 nfmark6 0 nettype 0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ipsec0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 16260 qdisc noqueue group 0x05 nfmark 0x200 nfmark6 0x200 nettype 0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 3e:e1:22:e5:27:ce brd ff:ff:ff:ff:ff:ff
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
4: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN mode DEFAULT group default qlen 1000
    link/tunnel6 :: brd ::
5: Port1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x02 nfmark 0x8002 nfmark6 0 nettype 0x02 state UP mode DEFAULT group default qlen 1000
    link/ether 00:50:56:b0:b8:96 brd ff:ff:ff:ff:ff:ff
6: Port2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x01 nfmark 0 nfmark6 0 nettype 0x01 state UP mode DEFAULT group default qlen 1000
    link/ether 00:50:56:b0:7d:c6 brd ff:ff:ff:ff:ff:ff
7: Port3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x01 nfmark 0 nfmark6 0 nettype 0x01 state UP mode DEFAULT group default qlen 1000
    link/ether 00:50:56:b0:de:e2 brd ff:ff:ff:ff:ff:ff
8: Port4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:50:56:b0:08:8c brd ff:ff:ff:ff:ff:ff`

$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue group 0x04 nfmark 0 nfmark6 0 nettype 0 state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ipsec0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 16260 qdisc noqueue group 0x05 nfmark 0x200 nfmark6 0x200 nettype 0 state UNKNOWN group default qlen 1000
    link/ether 3e:e1:22:e5:27:ce brd ff:ff:ff:ff:ff:ff
    inet 169.254.234.5/32 scope global ipsec0
       valid_lft forever preferred_lft forever
    inet6 fe80::3ce1:22ff:fee5:27ce/64 scope link 
       valid_lft forever preferred_lft forever
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
4: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN group default qlen 1000
    link/tunnel6 :: brd ::
5: Port1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x02 nfmark 0x8002 nfmark6 0 nettype 0x02 state UP group default qlen 1000
    link/ether 00:50:56:b0:b8:96 brd ff:ff:ff:ff:ff:ff
    inet 10.170.1.119/20 brd 10.170.15.255 scope global Port1
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:feb0:b896/64 scope link 
       valid_lft forever preferred_lft forever
6: Port2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x01 nfmark 0 nfmark6 0 nettype 0x01 state UP group default qlen 1000
    link/ether 00:50:56:b0:7d:c6 brd ff:ff:ff:ff:ff:ff
    inet 222.222.221.11/24 brd 222.222.221.255 scope global Port2
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:feb0:7dc6/64 scope link 
       valid_lft forever preferred_lft forever
7: Port3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast group 0x01 nfmark 0 nfmark6 0 nettype 0x01 state UP group default qlen 1000
    link/ether 00:50:56:b0:de:e2 brd ff:ff:ff:ff:ff:ff
    inet 222.222.223.11/24 brd 222.222.223.255 scope global Port3
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:feb0:dee2/64 scope link 
       valid_lft forever preferred_lft forever
8: Port4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop group 0 nfmark 0 nfmark6 0 nettype 0 state DOWN group default qlen 1000
    link/ether 00:50:56:b0:08:8c brd ff:ff:ff:ff:ff:ff`

3. Start staticd with below sample config

SF01V_SO01_SFOS 20.0.0 EAP0-Build2571# cat /conf/routing/staticd.conf
!
! Zebra configuration saved from vty
!   2023/03/16 16:21:18
!
frr version 8.4.2
frr defaults traditional
!
hostname static
log stdout
!
debug static events
debug static route
!
!
ip route 3.0.0.0/8 222.222.221.11 Port2
ip route 10.171.0.0/20 10.170.1.1 Port1
ip route 10.167.0.44/32 10.170.1.1 Port1
ip route 10.171.1.239/32 10.170.1.1 Port1
ip route 10.171.64.3/32 10.170.1.1 Port1
ip route 10.171.64.4/32 10.170.1.1 Port1
ip route 10.171.64.110/32 10.170.1.1 Port1
ip route 10.171.64.111/32 10.170.1.1 Port1
ip route 10.171.64.112/32 10.170.1.1 Port1
ip route 10.171.66.140/32 10.170.1.1 Port1
ip route 10.171.68.140/32 10.170.1.1 Port1
!
line vty
 no login
exit
!

4. Now try to add route ip route 4.4.4.0/24 222.222.221.11 Port2

Output:

static(config)# show running-config 

Current configuration:
!
frr version 8.4.2
frr defaults traditional
!
hostname static
log stdout
!
debug static events
debug static route
!
!
ip route 10.171.0.0/20 10.170.1.1 Port1
ip route 4.4.4.0/24 222.222.221.11 Port2
ip route 10.167.0.44/32 10.170.1.1 Port1
ip route 10.171.1.239/32 10.170.1.1 Port1
ip route 10.171.64.3/32 10.170.1.1 Port1
ip route 10.171.64.4/32 10.170.1.1 Port1
ip route 10.171.64.110/32 10.170.1.1 Port1
ip route 10.171.64.111/32 10.170.1.1 Port1
ip route 10.171.64.112/32 10.170.1.1 Port1
ip route 10.171.66.140/32 10.170.1.1 Port1
ip route 10.171.68.140/32 10.170.1.1 Port1
!
line vty
 no login
exit
!
end

5. Kernel route table

$ ip route show
3.0.0.0/8 via 222.222.221.11 dev Port2 proto 196 metric 20 
10.167.0.44 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.170.0.0/20 dev Port1 proto kernel scope link src 10.170.1.119 
10.171.0.0/20 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.1.239 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.64.3 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.64.4 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.64.110 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.64.111 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.64.112 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.66.140 via 10.170.1.1 dev Port1 proto 196 metric 20 
10.171.68.140 via 10.170.1.1 dev Port1 proto 196 metric 20 
222.222.221.0/24 dev Port2 proto kernel scope link src 222.222.221.11 
222.222.223.0/24 dev Port3 proto kernel scope link src 222.222.223.11

Expected behavior

  1. Route ip route 4.4.4.0/24 222.222.221.11 Port2 should be successfully added to kernel

Versions

  • OS Version: Linux build from scratch (openwrt based build)
  • Kernel: Linux 4.14.277
  • FRR Version: 8.3.0
  • FRR commit: 47ed380ba4febc72259ea68c397368beb059d32a

punith-shivakumar avatar Mar 16 '23 21:03 punith-shivakumar

Probably issue started with change https://github.com/FRRouting/frr/pull/9674/commits/f1d6b7e36e7e50c94ac180dcd594154b8c5812b4

punith-shivakumar avatar Mar 16 '23 21:03 punith-shivakumar

Hi Punith,

Are you using '-X' option for zebra by any chance ? Could you please share the deamon file o/p ?

rgirada avatar Mar 17 '23 13:03 rgirada

Hi Punith,

Are you using '-X' option for zebra by any chance ? Could you please share the deamon file o/p ?

  • I do not know what is 'X' option. Just running simple options zebra -f /conf/routing/zebra.conf -z @tcp4 -u nobody -g nobody -P 2709

  • I found actual change that caused failure https://github.com/FRRouting/frr/commit/b1ab2876fa8d02fbb84feafde8e321456409ec1a

  • Only first route is allowed (ip route 3.0.0.0/8 222.222.221.11 Port2) and starts failing from second route ( ip route 4.4.4.0/24 222.222.221.11 Port2 )

  • If first route is removed and then adding second one succeed

  • Reverting the change mentioned above allowed multiple routes to add successfully

punith-shivakumar avatar Mar 19 '23 04:03 punith-shivakumar

It seems like the actual bug here is that staticd is still allowing a route with a nexthop set to a connected address to be installed at all, which is what https://github.com/FRRouting/frr/commit/b1ab2876fa8d02fbb84feafde8e321456409ec1a was supposed to prevent.

@rgirada thoughts?

qlyoung avatar Jul 24 '23 21:07 qlyoung

This issue is stale because it has been open 180 days with no activity. Comment or remove the autoclose label in order to avoid having this issue closed.

github-actions[bot] avatar Jan 21 '24 01:01 github-actions[bot]

This issue will be automatically closed in the specified period unless there is further activity.

frrbot[bot] avatar Jan 21 '24 01:01 frrbot[bot]

It seems like the actual bug here is that staticd is still allowing a route with a nexthop set to a connected address to be installed at all, which is what b1ab287 was supposed to prevent.

@rgirada thoughts?

Any further update?

punith-shivakumar avatar Jan 22 '24 02:01 punith-shivakumar

This issue will no longer be automatically closed.

frrbot[bot] avatar Jan 22 '24 02:01 frrbot[bot]

It seems like the actual bug here is that staticd is still allowing a route with a nexthop set to a connected address to be installed at all, which is what b1ab287 was supposed to prevent.

Why would you prevent that? What else are you supposed to use as the next hop for a static?

amuckart avatar Jun 10 '24 05:06 amuckart

When I tried to add the IP route 4.4.4.0/24 222.222.221.11 Port2 to the kernel, it was successfully added when I tested the aforementioned setup. How can I replicate this bug?

Darwin4053 avatar Jul 30 '24 09:07 Darwin4053