Aleksandr Sorokin

> Did you have ctlb disabled before upgrade? Nope Can you provide additional parameters which I need to set up? I think it is something with bpfConnectTimeLoadBalancingEnabled bpfConnectTimeLoadBalancing bpfHostNetworkedNATWithoutCTLB It's...

> > killing calico-node pod immediately fixing the problem > > What changes after killing the pod. Could you share your routing table before/after? Routes almost did not change I...

> I wonder is some routes caching is in play. Could you dump `ip route show cached` ? `ip route show cached` it is empty before and after killing the...

> Just to confirm, do you see the same problem from host-networked pods/processes or from regular pods as well? Regular pods don't have a problem. They work fine. Tested it...

> Would you be able to tcpdump whether your traffic is reaching the service, what kind of packets are exiting from `bpfout.cali` on your test node? [bpfout.cali.pcap.gz](https://github.com/user-attachments/files/15789919/bpfout.cali.pcap.gz) So fresh vm...

> `10.243.0.10` is a local pod or remote? it's a service IP pods behind that IP are remote hostNetwork -> pod IP has no issue

> bpfLogLevel: Debug > bpfLogFilters: > - all: host 172.24.1.29 and udp port 53 that does not work these changes have been accepted by API ```yaml bpfLogLevel: Debug bpfLogFilters: all:...

BTW I found a repeated error in the tigera operator probably not related to this issue ``` {"level":"error","ts":"2024-06-12T08:07:07Z","logger":"controller_ippool","msg":"Cannot update an IP pool not owned by the operator","Request.Namespace":"","Request.Name":"periodic-5m0s-reconcile-event","reason":"ResourceValidationError","stacktrace":"github.com/tigera/operator/pkg/controller/status.(*statusManager).SetDegraded\n\t/go/src/github.com/tigera/operator/pkg/controller/status/status.go:356\ngithub.com/tigera/operator/pkg/controller/ippool.(*Reconciler).Reconcile\n\t/go/src/github.com/tigera/operator/pkg/controller/ippool/pool_controller.go:291\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:118\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:314\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226"} ```

## Before pod killing (after node restart when we have the problem) ```bash cat /proc/sys/net/ipv4/conf/bpfout.cali/rp_filter 1 ``` route ```bash route -n Kernel IP routing table Destination Gateway Genmask Flags Metric...

> What is your linux distro (which I should have asked a while ago)? ``` # uname -r 5.14.0-452.el9.x86_64 # cat /etc/os-release NAME="CentOS Stream" VERSION="9" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="9" PLATFORM_ID="platform:el9"...