frr icon indicating copy to clipboard operation
frr copied to clipboard

Split Default-Route not being advertised via BGP

Open sunch4se opened this issue 3 years ago • 14 comments

FRR 8.2, Ubuntu 18.04.6 LTS, 4.15.0-191-generic

Describe the bug Advertising a "split" default-route of 0.0.0.0/1 and 128.0.0.0/1 through a route-map only ends up with route 128.0.0.0/1 being advertised.

[ x] Did you check if this is a duplicate issue? [ ] Did you test it on the latest FRRouting/frr master branch?

Expected behavior Prefixes 0.0.0.0/1 and 128.0.0.0/1 should be advertised.

Configuration route-map:

route-map rm-outbound-nlc-prepend-as permit 200
 description prepend 4 x AS 65153, permit drsplit
 match ip address prefix-list drsplit
 set as-path prepend 65153 65153 65153 65153
exit
!
route-map rm-outbound-nlc-prepend-as permit 300
 description prepend 4 x AS 65153, permit policyroutes
 match ip address prefix-list policyrouting
 set as-path prepend 65153 65153 65153 65153
exit
!

prefix-lists:

ip prefix-list drsplit seq 5 permit 0.0.0.0/1
ip prefix-list drsplit seq 10 permit 128.0.0.0/1
ip prefix-list drsplit seq 9999 deny any
ip prefix-list policyrouting seq 100 permit 10.168.10.64/32
ip prefix-list policyrouting seq 200 permit 172.16.60.163/32
ip prefix-list policyrouting seq 300 permit 172.16.248.73/32

routes:

ip route 0.0.0.0/1 10.168.16.110 120
ip route 128.0.0.0/1 10.168.16.110 120

Behavior: advertised routes:

 show ip bgp neighbors 10.139.32.102 advertised-routes
BGP table version is 2366, local router ID is 10.168.16.21, vrf id 0
Default local pref 100, local AS 65153
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.168.10.64/32  0.0.0.0                  0         32768 65153 65153 65153 65153 ?
*> 128.0.0.0/1      0.0.0.0                  0         32768 65153 65153 65153 65153 ?
*> 172.16.60.163/32 0.0.0.0                  0         32768 65153 65153 65153 65153 ?
*> 172.16.248.73/32 0.0.0.0                  0         32768 65153 65153 65153 65153 ?

Total number of prefixes 4

show ip bgp:

show ip bgp 0.0.0.0/1
BGP routing table entry for 0.0.0.0/1, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  Local
    10.168.16.110 from 0.0.0.0 (10.168.16.21)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Aug 10 05:41:45 2022
show ip bgp 128.0.0.0/1
BGP routing table entry for 128.0.0.0/1, version 629
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  10.128.176.102 10.128.192.102 10.128.208.102 10.128.224.102 10.129.0.102 10.129.16.102 10.129.32.102 10.129.48.102 10.129.64.102 10.129.80.102 10.129.96.102 10.129.112.102 10.129.144.102 10.129.160.102 10.129.176.102 10.129.192.102 10.129.208.102 10.129.224.102 10.129.240.102 10.130.0.102 10.130.16.102 10.130.32.102 10.130.48.102 10.130.64.102 10.130.80.102 10.130.96.102 10.130.112.102 10.130.128.102 10.130.144.102 10.130.160.102 10.130.176.102 10.130.192.102 10.130.208.102 10.130.224.102 10.130.240.102 10.131.0.102 10.131.16.102 10.131.32.102 10.131.48.102 10.131.64.102 10.131.80.102 10.131.96.102 10.131.112.102 10.131.128.102 10.131.144.102 10.131.160.102 10.131.176.102 10.131.192.102 10.131.208.102 10.131.224.102 10.131.240.102 10.132.16.102 10.132.32.102 10.132.48.102 10.132.64.102 10.132.96.102 10.132.112.102 10.132.144.102 10.132.160.102 10.132.224.102 10.133.0.102 10.133.48.102 10.133.64.102 10.133.80.102 10.133.112.102 10.133.128.102 10.133.144.102 10.133.160.102 10.133.176.102 10.133.192.102 10.133.208.102 10.133.224.102 10.133.240.102 10.134.0.102 10.134.16.102 10.134.32.102 10.134.64.102 10.134.80.102 10.134.96.102 10.134.112.102 10.134.128.102 10.134.144.102 10.134.160.102 10.134.176.102 10.134.192.102 10.134.208.102 10.134.224.102 10.134.240.102 10.135.0.102 10.135.16.102 10.135.32.102 10.135.48.102 10.135.64.102 10.135.80.102 10.135.112.102 10.135.128.102 10.135.160.102 10.135.176.102 10.135.192.102 10.135.208.102 10.135.224.102 10.135.240.102 10.136.0.102 10.136.16.102 10.136.32.102 10.136.48.102 10.136.64.102 10.136.80.102 10.136.96.102 10.136.128.102 10.136.144.102 10.136.160.102 10.136.192.102 10.136.208.102 10.138.64.102 10.139.32.102 10.163.176.102 10.168.16.1
  Local
    10.168.16.110 from 0.0.0.0 (10.168.16.21)
      Origin incomplete, metric 0, weight 32768, valid, sourced, best (First path received)
      Last update: Wed Aug 10 05:41:44 2022

Comment: Adding only 0.0.0.0/0 to the prefix-list ends up with no route being advertised at all.

sunch4se avatar Sep 12 '22 13:09 sunch4se

Please show a full config. But from what I see, you need on-math continue statement under route-map rm-outbound-nlc-prepend-as permit 200 in order to evaluate 300 rule.

ton31337 avatar Sep 12 '22 16:09 ton31337

Hi,

unfortunately that's not helping. In fact, this is causing the 128.0.0.0/1 prefix entry to also not get advertised. And the 200 rule is indeed the one that is not working correctly. So it is probably no point to evaluate rule 300 anyway?

Here's the full config without neighbor-statements and routes (hope that's enough):

!
frr version 8.3
frr defaults traditional
service integrated-vtysh-config
!
!
ip route 0.0.0.0/1 10.168.16.110 120
ip route 128.0.0.0/1 10.168.16.110 120
some more routes ...
!
interface lo1
 ip address 10.168.16.154/32
exit
!
interface ens160
 ip ospf authentication
 ip ospf authentication-key
exit
!
router bgp 65153
 bgp router-id 10.168.16.21
 no bgp ebgp-requires-policy
 no bgp suppress-duplicates
 no bgp hard-administrative-reset
 no bgp graceful-restart notification
 no bgp network import-check
 neighbors ...
 !
 address-family ipv4 unicast
  redistribute static route-map static2bgp
  redistribute ospf route-map ospf2bgp
  neighbor 10.139.32.102 soft-reconfiguration inbound
  neighbor 10.139.32.102 weight 32768
  neighbor 10.139.32.102 route-map rm-inbound-setnexthop in
  neighbor 10.139.32.102 route-map rm-outbound-nlc-prepend-as-new out
  some more neighbors ...
 exit-address-family
exit
!
router ospf
 ospf router-id 10.168.16.21
 redistribute bgp route-map bgp2ospf
 graceful-restart
 graceful-restart helper enable
 network 10.168.16.0/24 area 10.168.16.0
 area 10.168.16.0 authentication
exit
!
!
ip prefix-list drsplit seq 5 permit 0.0.0.0/1
ip prefix-list drsplit seq 10 permit 128.0.0.0/1
ip prefix-list drsplit seq 9999 deny any
ip prefix-list policyrouting seq 100 permit 10.168.10.64/32
ip prefix-list policyrouting seq 200 permit 172.16.60.163/32
ip prefix-list policyrouting seq 300 permit 172.16.248.73/32
ip prefix-list drsplit-new seq 5 permit 0.0.0.0/1
ip prefix-list drsplit-new seq 10 permit 128.0.0.0/1
ip prefix-list drsplit-new seq 9999 deny any
some more prefix-lists ...
!
!
route-map bgp2ospf permit 10
 a few more entries..
!
route-map ospf2bgp permit 10 
 a few more entries
!
route-map rm-inbound-setnexthop deny 100
 match ip address prefix-list cmk
exit
!
route-map rm-inbound-setnexthop permit 200
 description "next hop is ipsec-markt2"
 set ip next-hop 10.168.16.130
exit
!
route-map rm-outbound-denypolicyroutes deny 100
 description "do not redist to mpls"
 match ip address prefix-list policyrouting
exit
!
route-map rm-outbound-denypolicyroutes permit 200
 description "permit all"
exit
!
route-map rm-outbound-nlc-prepend-as permit 200
 description prepend 4 x AS 65153, permit drsplit
 match ip address prefix-list drsplit
 set as-path prepend 65153 65153 65153 65153
exit
!
route-map rm-outbound-nlc-prepend-as permit 300
 description prepend 4 x AS 65153, permit policyroutes
 match ip address prefix-list policyrouting
 set as-path prepend 65153 65153 65153 65153
exit
!
route-map static2bgp permit 10
 match ip address prefix-list drsplit
exit
!
route-map static2bgp permit 20
 match ip address prefix-list supernet-markt
exit
!
route-map static2bgp permit 30
 match ip address prefix-list policyrouting
exit
!

!
route-map rm-outbound-nlc-prepend-as-new permit 200
 description prepend 4 x AS 65153, permit drsplit-new
 match ip address prefix-list drsplit-new
 on-match next
 set as-path prepend 65153 65153 65153 65153
exit
!
route-map rm-outbound-nlc-prepend-as-new permit 300
 description prepend 4 x AS 65153, permit policyroutes
 match ip address prefix-list policyrouting
 set as-path prepend 65153 65153 65153 65153
exit
!
end

sunch4se avatar Sep 12 '22 18:09 sunch4se

Please show "show ip route".

ton31337 avatar Sep 13 '22 08:09 ton31337

Those should be the relevant routes: O 0.0.0.0/0 [110/10] via 10.168.16.110, ens160, weight 1, 04w6d02h K>* 0.0.0.0/0 [0/0] via 10.168.16.110, ens160, 04w6d02h S>* 0.0.0.0/1 [120/0] via 10.168.16.110, ens160, weight 1, 04w6d02h S>* 128.0.0.0/1 [120/0] via 10.168.16.110, ens160, weight 1, 04w6d02h

There are of course more OSPF and static-routes, but those give should hopefully enough.

On a side-note: We have updated from 7.2 release - with 7.2 this was all working as expected.

sunch4se avatar Sep 13 '22 08:09 sunch4se

I tested with your given config, I can't replicate this. Can you enable debug bgp updates and see why 0.0.0.0/1 is denied in the logs?

ton31337 avatar Sep 15 '22 18:09 ton31337

I can't get debugging fixed properly at the moment, but I assume I found some more info why this is not working:

show bgp ipv4 route-map rm-outbound-nlc-prepend-as
BGP table version is 781, local router ID is 10.168.16.21, vrf id 0
Default local pref 100, local AS 65153
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

   Network          Next Hop            Metric LocPrf Weight Path
*  0.0.0.0/1        10.168.16.110            0         32768 ?
*> 10.168.10.64/32  10.168.16.110            0         32768 ?
*> 128.0.0.0/1      10.168.16.110            0         32768 ?
*> 172.16.60.163/32 10.168.16.110            0         32768 ?
*> 172.16.248.73/32 10.168.16.110            0         32768 ?

Displayed  5 routes and 1017 total paths

z016nrt0310# show ip route 0.0.0.0/1
Routing entry for 0.0.0.0/1
  Known via "static", distance 120, metric 0, best
  Last update 11:35:35 ago
  * 10.168.16.110, via ens160, weight 1
z016nrt0310# show ip bgp 0.0.0.0/1
BGP routing table entry for 0.0.0.0/1, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  Local
    10.168.16.110 from 0.0.0.0 (10.168.16.21)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Sun Sep 18 19:22:01 2022
z016nrt0310#
z016nrt0310# show ip route kernel
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

K>* 0.0.0.0/0 [0/0] via 10.168.16.110, ens160, 11:39:24

I'm not sure what the issue is though, since this has been working flawlessly in past versions.

sunch4se avatar Sep 19 '22 06:09 sunch4se

If you could enable logs, it would be much easier to spot why it's not added to BGP.

ton31337 avatar Sep 19 '22 09:09 ton31337

This is a debug from the last few hours. Unfortunately I cannot see anything related to prefix 0.0.0.0/1.

2022/09/19 10:54:16 BGP: [HVRWP-5R9NQ] u2:s45 send UPDATE 10.168.10.64/32 IPv4 unicast
2022/09/19 10:54:16 BGP: [HVRWP-5R9NQ] u2:s45 send UPDATE 172.16.248.73/32 IPv4 unicast
2022/09/19 10:54:16 BGP: [HVRWP-5R9NQ] u2:s45 send UPDATE 172.16.60.163/32 IPv4 unicast
2022/09/19 10:54:16 BGP: [HVRWP-5R9NQ] u2:s45 send UPDATE 128.0.0.0/1 IPv4 unicast

Other than that I can see lots of messages with the following:

2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.64.24.18/32 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.64.24.19/32 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.64.24.20/32 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.100.111.224/28 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.100.112.184/29 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.3.96/28 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.48.0/20 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.48.0/21 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.48.0/24 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.56.0/21 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.64.0/20 is filtered by route-map 'rm-outbound-nlc-prepend-as'
2022/09/19 11:27:20 BGP: [Q9J6Z-09HRR] 10.129.16.102 [Update:SEND] 10.128.64.0/21 is filtered by route-map 'rm-outbound-nlc-prepend-as'

Again, nothing related to 0.0.0.0/1 though.

sunch4se avatar Sep 19 '22 14:09 sunch4se

This smells like an issue with the route-map optimizations.

show route-map rm-outbound-nlc-prepend-as prefix I believe is the command to display the tree built for that map.

taspelund avatar Sep 19 '22 14:09 taspelund

Also, keep logging enabled and remove 0.0.0.0/1 then recreate it. Then show the logs here. From this output, https://github.com/FRRouting/frr/issues/11935#issuecomment-1250638560 seems that the route-map kinda works OK, but the route is not accepted.

ton31337 avatar Sep 19 '22 14:09 ton31337

You mean remove the route for 0.0.0.0/1 temporarily, or remove the prefix from the prefix-list? I think I also disabled rm-optimization, which unfortunately did not help.

sunch4se avatar Sep 19 '22 18:09 sunch4se

You mean remove the route for 0.0.0.0/1 temporarily, or remove the prefix from the prefix-list? I think I also disabled rm-optimization, which unfortunately did not help.

You can try both. Remove from prefix-list and/or static route.

ton31337 avatar Sep 19 '22 18:09 ton31337

The issue is fixed after removing and re-adding the route for 0.0.0.0/1. Removing and re-adding the prefix from the prefix-list did not resolve the issue. Now this here looks different also:

z016nrt0310# show ip bgp 0.0.0.0/1
BGP routing table entry for 0.0.0.0/1, version 808
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  **lots of bgp neighbors...*
  Local
    10.168.16.110 from 0.0.0.0 (10.168.16.21)
      Origin incomplete, metric 0, weight 32768, valid, sourced, best (First path received)
      Last update: Tue Sep 20 19:12:27 2022

sunch4se avatar Sep 20 '22 19:09 sunch4se

Are you able to replicate easily the same behavior? Logs still would be useful.

ton31337 avatar Sep 21 '22 16:09 ton31337