Unable to establish connection between GK and GT

[nimrodkr]

GK and GT on KVM using virtio with multiple virtual networks, Attached topology map of the setup After starting both GK & GT, initiating bird and having BGP established Remote network apears in GK route When calling: local ret = dylib.c.add_fib_entry("198.51.100.0/24", "10.0.2.2", nil, dylib.c.GK_FWD_GRANTOR, dyc.gk) I am getting (from log):

GATEKEEPER: lpm: IPv4 lookup miss for 198.51.100.0
GATEKEEPER GK: Prefix lookup did not find existing neighbor FIB on front interface, as expected
GATEKEEPER: lpm: IPv4 lookup miss for 198.51.100.0
GATEKEEPER GK: Prefix lookup did not find existing neighbor FIB on back interface, as expected
GATEKEEPER DYN CFG: Client disconnected

Any ideas what I am doing wrong?

[AltraMayor]

The log entries you singled out are not errors, but sanity checks. Gatekeeper does a number of verifications on each entry that goes into its routing table to hopefully reject configuration errors that can lead to operational disasters.

The issue with your setup is a missing router between the Gatekeeper server, the Grantor server, and the destination server. Grantor servers do not route packets, they only deal with local networks. In your example, once the Grantor decapsulates a packet, it notices that 198.51.100.X is not local to its network (i.e. 10.0.2.2/24), so it redirects the decapsulated packet to 10.0.1.1 (from where the packet came) in the hope it's a router. Once you insert the router in the topology, you'll have the third parameter to pass to dylib.c.add_fib_entry(). Grantor servers are either located in the same LAN as the destination servers, or somewhere else connected to a router.

[nimrodkr]

First, Thanks for your quick response. I have made the changes and got to the point where the FIB entry is deployed and traffic from client arrives to the kni_front . However I can't see any traffic on the kni_back (expecting to see the IPinIP encap). The GK dump is: gk: successfully processed all the FIB entries FIB entry for IP prefix: 10.0.5.0/24 with action FWD_GRANTOR (0) Grantor IP address: 10.0.5.2 Ethernet cache entry: [state: fresh, nexthop ip: 10.0.3.2, d_addr: 52:54:00:0B:E1:F5] Neighbor Ethernet cache entry: [state: fresh, neighbor ip: 10.0.3.2, d_addr: 52:54:00:0B:E1:F5, action: FWD_NEIGHBOR_BACK_NET (4)] Any thought of what I am doing wrong? (attached a picture of the new topology) Thanks again :)

[cjdoucette]

Note that kni_front and kni_back are virtual interfaces that Gatekeeper creates only for the purpose of communicating with a routing daemon in Linux, so you won't see any data plane traffic (i.e. the encapsulated traffic being sent to Grantor) on these interfaces. Only BGP traffic (or other control plane traffic) uses them.

It's not currently possible to capture the traffic on a DPDK interface like you would with tcpdump on a Linux interface. Well, it is, but Gatekeeper I believe would need to be patched to provide support for this [1].

To verify packets are flowing through Gatekeeper as intended, you can inspect the GK block's measurement statistics that are logged or do a tcpdump on the R1's front interface.

[1] https://doc.dpdk.org/guides/howto/packet_capture_framework.html

[AltraMayor]

As @cjdoucette suggested, capture packets at the routers' interfaces. Otherwise, your configuration seems fine.

[nimrodkr]

Hi, I have done the capture on the router itself (they are Ubuntu 18.04 running Quagga daemon). So I can see traffic progressing from R1 to the GK interface, but nothing comes out. Is there an initial policy I must place after/before setting the FIB? Thanks...

[AltraMayor]

Did you capture traffic from both router interfaces? If a packet goes to the front of the Gatekeeper server and it's forwarded to the Grantor server it goes through the back interface, not back in the front interface.

Once the FIB entry that forwards to a Grantor server is set, Gatekeeper servers will forward the packets. No policy or Grantor server is needed for this; the policy and Grantor server will be needed once the encapsulated packets arrive at the Grantor server.

There's no much information in your comments to enable us to guide you further. My general suggestion is for you to stop the Gatekeeper server and Grantor server, and check the connectivity of your setup. There's a couple of things that can go wrong in the configuration of the two routers, for example (it's not an exhaustive list):

1. R1 can reach 10.0.5.0/24 through the front interface of the Gatekeeper server and through the interface that goes to the second router. If nothing is done, the most likely outcome is that the second link is being ignored. That is, packets coming from the back interface of the Gatekeeper server would be redirected back to the front interface.
2. The second router must be aware of the prefix 10.0.1.0/24 and 10.0.3.0/24; otherwise, the destination server won't be able to reach back the client and the Grantor server won't be able to notify the Gatekeeper server of policy decisions.

Once the connectivity is validated, you bring back the Gatekeeper and Grantor server. Besides tracking packets, log messages can give important clues to find out what's going on.

[AltraMayor]

Notice that issues have been identified while running Gatekeeper on KVM; see issue #464 for details.