"Cilium bgp routes" not showing received routes

[arcreigh]

Is there an existing issue for this?

• [X] I have searched the existing issues

Version

higher than v1.16.0 and lower than v1.17.0

What happened?

I have a fortigate advertising 26 routes to my 6 node kubeadm cluster.

all peers are established and each show that they have recevied 26 routes and advertised 2 routes.

however when I run cilium bgp routes it only shows the advertised routes and not the received routes.

How can we reproduce the issue?

run cilium bgp routes

Cilium Version

cilium-cli: v0.16.15 compiled with go1.22.5 on linux/amd64 cilium image (default): v1.16.0 cilium image (stable): v1.16.0 cilium image (running): 1.16.0

Kernel Version

Linux k8s-master-1 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version

Client Version: v1.26.4+vmware.wcp.0 Kustomize Version: v4.5.7 Server Version: v1.29.3

Regression

n/a / unknown

Sysdump

cilium-sysdump-20240811-180942.zip

Relevant log output

No response

Anything else?

No response

Cilium Users Document

• [ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[rastislavs]

Hi, ATM, BGP Control Plane in Cilium can be used only for advertising Cilium-specific routes to the upstream routers, Cilium is not installing any received routes on the Cilium nodes. Because of that, they are not displayed in cilium bgp routes output.

[arcreigh]

Hi,

ATM, BGP Control Plane in Cilium can be used only for advertising Cilium-specific routes to the upstream routers, Cilium is not installing any received routes on the Cilium nodes. Because of that, they are not displayed in cilium bgp routes output.

Even though it may not be installing routes, from a troubleshooting perspective I think it would be important to at least see what is being received and does open the start to adding the installation of routes.

[rastislavs]

Yeah, I get the troubleshooting perspective. The cilium bgp routes is a shortcut to cilium bgp routes available, which reflect's GoBGP's main RIB. The received routes should not appear there, until we enable route installation on Cilium (ATM received routes never get there thanks to the default reject import policy introduced in https://github.com/cilium/cilium/pull/33035).

Then we have cilium bgp routes advertised, which reflects GoBGP's adj-rib-out.

What would you like to see for debugging purposes is GoBGP's adj-rib-in, so for that something like cilium bgp routes received could be introduced. I think we will definitely need it if/once we implement route installation on Cilium.

[arcreigh]

I'm not too familiar with goBGP, is there a one liner show command for the RIB to see what is being received?

I understand that this is a still in active development area of cilium, I would recommend trying to be in line with Cisco as far as command vernacular goes as they tend to be the market leader when it comes to routing. Matching certain commands and detailed output helps us network folk align with the server folks now that the two worlds are coming together in a lot of ways.

Switching from cilium bgp peers to cilium bgp neighbors as an example.

This would be an immense help when it comes to cross communication between a k8s focused team and the network team as they can all speak the same language so to speak when asking for specific command output for troubleshooting purposes.

I also imagine at some point route-maps/filters will be a thing.

[rastislavs]

I'm not too familiar with goBGP, is there a one liner show command for the RIB to see what is being received?

There is a gobgp CLI command for that, however not yet exposed on Cilium

Switching from cilium bgp peers to cilium bgp neighbors as an example.

Both alternatives work actually ;)

Yeah, thanks for the suggestions, I think they are all valid.

[yushoyamaguchi]

@rastislavs

If I implement the sub-command for confirm adj-rib-in, ( like cilium bgp routes received), do you have intention to merge? If you have, I want to implement it. Maybe, it won't take that long...

CC @arcreigh

[dswaffordcw]

Hi @arcreigh,

I am the contributor who recently changed this in Cilium's CLI. Prior to https://github.com/cilium/cilium/pull/33035, cilium bgp routes did show all received routes. I modified this for a few reasons, one being that it's confusing for us to display routes when they have no effect. As @rastislavs pointed out, Cilium is unable to install routes learned by BGP today.

The output of cilium bgp peers shows accepted/imported routes. Cilium is applying a default IMPORT NONE style route map across all peers.

I'd like to understand a bit more how viewing received routes would be helpful to you? As they have no effect, you may want to consider a route map on the remote peer which blocks announcements toward Cilium.

To your other question, there is a way to see routing policies configured in Cilium but not from the cilium command. These are visible from the cilium-dbg command. That is a CLI installed into the Cilium containers on a given Kubernetes node. To see that, try this out:

1. Run kubectl get pods -n kube-system --selector=app.kubernetes.io/part-of=cilium
2. Locate Cilium pods named cilium-<hash>
3. Run kubectl exec -n kube-system cilium-<hash> cilium-dbg bgp route-policies

[arcreigh]

Hey there @dswaffordcw!

As a network engineer we tend to pre-configure the networking environment before projects / applications are onboarded and set into a running state. So while Cilium today might not make use of the routes it receives or even install them. The network engineer in me would still like to know that once it does, there is nothing that I have to do on my end. I can see from cilium that they are receiving the routes they need for the future. That is good enough for me from a planning standpoint and it allows me to move on to the next thing.

The need is purely from a logistical and future planning standpoint.

[fredgnucc]

Hello guys just wondering is there a version available to install the route-received ? or something in the pipeline that will come soon !?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[github-actions[bot]]

This issue has not seen any activity since it was marked stale. Closing.