ipv6 ospf not rebuilding after interface reconnect ?

[mhazan01]

Hello all,

FRR version : 8.1-1ubuntu1.5 (package) OS : Ubuntu 22.04.3 LTS Kernel : 5.15.0-79-generic

i have some weird issue happening on my setup. 3 ubuntu host, each with a dual link interface connected in a ring and using frr ipv6 ospf to achieve link failure protection. that work great when i unplug the cable, the packets goes through the other interface and reach each host without issue. however, when i apply system upgrade that, i assume, restart both interfaces, even after the update process is ran, the os is unable to reach the neighbors and instead use the os default route (not on the same physical interface) and thus fail to ping. i have to issue a clear ipv6 ospf6 process command in the vtysh to get the route working again. the other host are configured the same (bare interface names, thanks stupid os) and have the same issue that happen when upgrading the OS.

host config : ethernets: lo: addresses: #loopback used for ceph cluster network - fc00::1/128 enp66s0f0: #10gb 1 configure with FRR dhcp4: false dhcp6: false mtu: 9000 #? link-local: []

enp66s0f1:
  #10gb 2 configured with FRR
  dhcp4: false
  dhcp6: false
  mtu: 9000

#? link-local: []

various cmd from the broken host 0.0.0.1 :

sh ipv6 route seem to indicate it knows where to find host, but ping doesn't work (goes to system default route)

O>r fc00::2/128 [110/20] via fe80::96f1:28ff:xx, enp66s0f1, weight 1, 01w5d19h O>r fc00::3/128 [110/20] via fe80::262:bff:xx, enp66s0f0, weight 1, 01w5d19h

ceph-01# sh run Building configuration...

Current configuration: ! frr version 8.1 frr defaults traditional hostname ceph-01 log syslog informational service integrated-vtysh-config ! interface enp66s0f0 ipv6 ospf6 area 0 ipv6 ospf6 network point-to-point exit ! interface enp66s0f1 ipv6 ospf6 area 0 ipv6 ospf6 network point-to-point exit ! interface lo ipv6 ospf6 area 0 exit ! router ospf6 ospf6 router-id 0.0.0.1 log-adjacency-changes exit ! end

ceph-01# sh ip ospf6 neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 0.0.0.3 1 00:00:38 Full/PointToPoint 12d19:39:39 enp66s0f0[PointToPoin t] 0.0.0.2 1 00:00:38 Full/PointToPoint 12d19:43:23 enp66s0f1[PointToPoin t] ceph-01# sh ip ospf6 neighbor Neighbor ID Pri DeadTime State/IfState Duration I/F[State] 0.0.0.3 1 00:00:32 Full/PointToPoint 12d19:39:56 enp66s0f0[PointToPoint] 0.0.0.2 1 00:00:32 Full/PointToPoint 12d19:43:40 enp66s0f1[PointToPoint] ceph-01# sh ip ospf6 database

    Area Scoped Link State Database (Area 0)

Type LSId AdvRouter Age SeqNum Payload Rtr 0.0.0.0 0.0.0.1 620 800007e4 0.0.0.3/0.0.0.5 Rtr 0.0.0.0 0.0.0.1 620 800007e4 0.0.0.2/0.0.0.6 Rtr 0.0.0.0 0.0.0.2 626 800007b4 0.0.0.1/0.0.0.7 Rtr 0.0.0.0 0.0.0.2 626 800007b4 0.0.0.3/0.0.0.4 Rtr 0.0.0.0 0.0.0.3 621 800007ed 0.0.0.2/0.0.0.7 Rtr 0.0.0.0 0.0.0.3 621 800007ed 0.0.0.1/0.0.0.2 INP 0.0.0.0 0.0.0.1 620 800007df fc00::1/128 INP 0.0.0.0 0.0.0.2 626 800007b0 fc00::2/128 INP 0.0.0.0 0.0.0.3 621 800007e6 fc00::3/128

    I/F Scoped Link State Database (I/F enp66s0f0 in Area 0)

Type LSId AdvRouter Age SeqNum Payload Lnk 0.0.0.2 0.0.0.1 1350 800007d6 fe80::2e76:8affxx Lnk 0.0.0.5 0.0.0.3 625 800007d6 fe80::262:bff:xx

    I/F Scoped Link State Database (I/F enp66s0f1 in Area 0)

Type LSId AdvRouter Age SeqNum Payload Lnk 0.0.0.7 0.0.0.1 1350 800007aa fe80::2e76:8aff:xx Lnk 0.0.0.6 0.0.0.2 840 800007a8 fe80::96f1:28ff:xx

    I/F Scoped Link State Database (I/F lo in Area 0)

Type LSId AdvRouter Age SeqNum Payload

    AS Scoped Link State Database

Type LSId AdvRouter Age SeqNum Payload

ceph-01# sh ip ospf6 interface bond0 is up, type BROADCAST Interface ID: 8 OSPF not enabled on this interface enp66s0f0 is up, type BROADCAST Interface ID: 2 Operating as type POINTOPOINT Internet Address: inet6: fe80::2e76:8aff:xx/64 Instance ID 0, Interface MTU 9000 (autodetect: 9000) MTU mismatch detection: enabled Area ID 0.0.0.0, Cost 10 State PointToPoint, Transmit Delay 1 sec, Priority 1 Timer intervals configured: Hello 10, Dead 40, Retransmit 5 DR: 0.0.0.0 BDR: 0.0.0.0 Number of I/F scoped LSAs is 2 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] 0 Pending LSAs for LSAck in Time 00:00:00 [thread off] enp66s0f1 is up, type BROADCAST Interface ID: 7 Operating as type POINTOPOINT Internet Address: inet6: fe80::2e76:8aff:xx/64 Instance ID 0, Interface MTU 9000 (autodetect: 9000) MTU mismatch detection: enabled Area ID 0.0.0.0, Cost 10 State PointToPoint, Transmit Delay 1 sec, Priority 1 Timer intervals configured: Hello 10, Dead 40, Retransmit 5 DR: 0.0.0.0 BDR: 0.0.0.0 Number of I/F scoped LSAs is 2 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] 0 Pending LSAs for LSAck in Time 00:00:00 [thread off] lo is up, type LOOPBACK Interface ID: 1 Internet Address: inet6: fc00::1/128 Instance ID 0, Interface MTU 65536 (autodetect: 65536) MTU mismatch detection: enabled Area ID 0.0.0.0, Cost 10 State Loopback, Transmit Delay 1 sec, Priority 1 Timer intervals configured: Hello 10, Dead 40, Retransmit 5 DR: 0.0.0.0 BDR: 0.0.0.0 Number of I/F scoped LSAs is 0 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] 0 Pending LSAs for LSAck in Time 00:00:00 [thread off]

ceph-01# show ipv6 ospf6 route detail Destination: fc00::1/128 Destination type: Network Installed Time: 12d23:58:06 ago Changed Time: 12d23:58:06 ago Lock: 2 Flags: BA-- Memory: prev: 0x0 this: 0x560c50ba35d0 next: 0x560c50ba4c90 Associated Area: 0.0.0.0 Path Type: Intra-Area LS Origin: Intra-Prefix Id: 0.0.0.0 Adv: 0.0.0.1 Options: --|-|-|--|-|-- Router Bits: -------- Prefix Options: --|--|--|--|-- Metric Type: 1 Metric: 10 (0) Paths count: 1 Nexthop count: 1 Nexthop: :: lo

Destination: fc00::2/128 Destination type: Network Installed Time: 12d19:49:39 ago Changed Time: 12d19:49:39 ago Lock: 2 Flags: BA-- Memory: prev: 0x560c50ba35d0 this: 0x560c50ba4c90 next: 0x560c50ba4240 Associated Area: 0.0.0.0 Path Type: Intra-Area LS Origin: Intra-Prefix Id: 0.0.0.0 Adv: 0.0.0.2 Options: --|-|-|--|-|-- Router Bits: -------- Prefix Options: --|--|--|--|-- Metric Type: 1 Metric: 20 (0) Paths count: 1 Nexthop count: 1 Nexthop: fe80::96f1:28ff:xx enp66s0f1

Destination: fc00::3/128 Destination type: Network Installed Time: 12d19:45:55 ago Changed Time: 12d19:45:55 ago Lock: 2 Flags: B--C Memory: prev: 0x560c50ba4c90 this: 0x560c50ba4240 next: 0x0 Associated Area: 0.0.0.0 Path Type: Intra-Area LS Origin: Intra-Prefix Id: 0.0.0.0 Adv: 0.0.0.3 Options: --|-|-|--|-|-- Router Bits: -------- Prefix Options: --|--|--|--|-- Metric Type: 1 Metric: 20 (0) Paths count: 1 Nexthop count: 1 Nexthop: fe80::262:bff:xx enp66s0f0

ceph-01# show ipv6 ospf6 spf-tree % Unknown command: show ipv6 ospf6 spf-tree ceph-01# show ipv6 ospf6 spf tree +-0.0.0.1 [0] +-0.0.0.2 [10] +-0.0.0.3 [10] ceph-01# show ipv6 ospf6 summary-address VRF Name: default aggregation delay interval :5(in seconds)

Summary-address Metric-type Metric Tag External_Rt_count

ceph-01# show ipv6 ospf6 OSPFv3 Routing Process (0) with Router-ID 0.0.0.1 Running 13d00:04:44 LSA minimum arrival 1000 msecs Maximum-paths 256 Initial SPF scheduling delay 0 millisec(s) Minimum hold time between consecutive SPFs 50 millsecond(s) Maximum hold time between consecutive SPFs 5000 millsecond(s) Hold time multiplier is currently 2 SPF algorithm last executed 12d19:52:29 ago, reason R+, R- Last SPF duration 0 sec 173 usec SPF timer is inactive Number of AS scoped LSAs is 0 Number of areas in this router is 1 Adjacency changes are logged

Area 0 Number of Area scoped LSAs is 6 Interface attached to this area: enp66s0f0 enp66s0f1 lo SPF last executed 1108349.290791s ago

that last line makes me think FRR is missing some even that would force SPF to refresh ?