AllowedIP without routing

[stevefan1999-personal]

Is your feature request related to a problem? Please describe.

I need to use Calico with my k0s cluster and I need to have allowed IP without auto IP route in netbird, because Calico already handled routing by using BIRD and will intelligently select wt0 as the gateway. But because my pod and service CIDR is not in the allowed IP list, I got required keys error which indeed is because the IP is not allowef

Describe the solution you'd like Allow the option to disable route but keep allowed IPs

Describe alternatives you've considered Use VXLAN on top of wt0, no dice Use another wireguard on top of wt0 (Calico supports wireguard too), cannot create IP socket

Additional context Add any other context or screenshots about the feature request here.

[ashish1099]

@stevefan1999-personal did you got it working ?

on my k8s cluster calico-node pod is just failing to start when netbird is running, and calico-node pods works fine, when netbird is stopped.

[stevefan1999-personal]

@ashish1099 I decided to remove Netbird from my K8S node simply because of this

[KlavsKlavsen]

Yeah.. we finally got time to start trying out netbird - but are stuck on this :(

[ashish1099]

This failed later on. I'm waiting on calico 3.26 which has a bgpfilter and will try to exclude the netbird cidr. will update here when I have reached that stage.

~Adding a bgpconfiguration which made this work.~

apiVersion: crd.projectcalico.org/v1
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Error
  nodeToNodeMeshEnabled: false
  asNumber: 64512

I got the idea from here https://github.com/projectcalico/calico/issues/3760#issuecomment-808418033

[damoasis]

Any news?

[ashish1099]

@damoasis the latest calico didn't worked for me unfortunately.

We are going to try cilium now and I will update it here, if I was able to made it work with netbird.

[mlsmaycon]

Hello @ashish1099, When creating an interface, NetBird sets its local route 100.X.X.X/16 to the host (or container) routing table. Is that the main issue, or is it happening when using network routes?

Can you share a bit more about the issue:

• Is the NetBird agent running on the host or as a container with the host network?
• Do you run the agent as a daemon on all nodes of the cluster?

[damoasis]

@mlsmaycon

• NetBird agent running on the host or as a container with the host network all has the same issue
• run the agent as a daemon on all nodes of the cluster

the test k8s environment (the cni is calico):

• 1 master,2 nodes(k8s-node1,k8s-node2), and 1 client (win 10 pc)
• run NetBird agent on all the nodes (include the master and the pc);
• the pod cidr is 10.233.0.0/16

In order to let the client connect to the pod with pod ip in the k8s cluster directly I add a Network route to k8s-node2:

• add Network Routes 10.233.0.0/16 to the peer k8s-node2 and the Distribution groups is the client

After add the network route:

• The client(win 10 pc) can connect to the pod in the k8s cluster with the pod ip directly
• And also the pod can connect to the client directly.

The problem is that:

• any pod on the k8s-node2 can not connet to other pod or service with service name or domain name but can still connect to other service or pod with ip address
• can not schedule any new pod to the k8s-node2
• and also the daemon set calico-node on the k8s-node2 in the kube-system namespace can not work any more when be killed. As a result the client could not connect to the pod any more.

But when remove the Routes and restart the machine of k8s-node2 all will go right.

[mlsmaycon]

@damoasis to make it work you removed the pod network 10.233.0.0/16 or the 100.X.X.X/16?

[damoasis]

@mlsmaycon

@damoasis to make it work you removed the pod network 10.233.0.0/16 or the 100.X.X.X/16?

10.233.0.0/16

[mlsmaycon]

Thanks for the update.

If the Kubernetes clients are configured as routers, they shouldn't be adding these routes. As you mentioned, the network route is distributing the routes to the client group. Can you confirm that the kubernetes nodes aren't part of the client group?

If possible, share the network route configuration in detail and group membership for your nodes.

[damoasis]

Thanks for the update.

If the Kubernetes clients are configured as routers, they shouldn't be adding these routes. As you mentioned, the network route is distributing the routes to the client group. Can you confirm that the kubernetes nodes aren't part of the client group?

If possible, share the network route configuration in detail and group membership for your nodes.

@mlsmaycon The kubernetes nodes aren't part of the client group, the client group onley contains the win 10 pc.

My requirement is to allow the win10 computer to directly access the IP of the internal pod in k8s through netbird.

[echowings]

I want this feature too! Is there a plan to implement it ?