Calico ebpf packet drops from tailscale (wireguard) to svc networks

[farcaller]

Expected Behavior

If ebpf is enabled, calico allows packets coming from a tailscale (a variant of wireguard) interface into services.

Current Behavior

calico drops such packets. Everything works as intended in iptables mode.

Possible Solution

N/A

Steps to Reproduce (for bugs)

1. A basic cluster of k3s with calico as CNI.
2. tailscale deployed on the node (the free one is sufficient), tailscale has --advertise-routes for the pod/svc CIDRs on the node.
3. Spin up e.g. nginx
4. Try accessing the pod from another machine in the tailnet.

Context

I want to access pods/services via the VPN link.

You can see the packets being rejected:

 .tailscaled-wra-3132    [007] D..2.  2537.834382: bpf_trace_printk: tailscal-I: New packet at ifindex=3; mark=0
 .tailscaled-wra-3132    [007] D..2.  2537.834384: bpf_trace_printk: tailscal-I: No metadata is shared by XDP
 .tailscaled-wra-3132    [007] D..2.  2537.834385: bpf_trace_printk: tailscal-I: IP id=49333 s=efc60000 d=a002
 .tailscaled-wra-3132    [007] D..2.  2537.834386: bpf_trace_printk: tailscal-I: Drop malformed IP packets
 .tailscaled-wra-3132    [007] D..2.  2537.834386: bpf_trace_printk: tailscal-I: Drop malformed or unsupported packet
 .tailscaled-wra-3132    [007] D..2.  2537.834387: bpf_trace_printk: tailscal-I: Final result=DENY (ec). Program execution time: 1614ns

Your Environment

• Calico version: v3.23.3
• Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes/k3s
• Operating System and version: nisox-22.05

[farcaller]

#4326 seems similar/related.

[farcaller]

Sample packet as seen from tcpdump's pov:

    100.81.82.37.16178 > 10.100.192.181.80: Flags [S], cksum 0x3d87 (correct), seq 4012059999, win 64240, options [mss 1460,sackOK,TS val 608374192 ecr 0,nop,wscale 7], length 0
        0x0000:  4500 003c e90f 4000 3f06 d11c 6451 5225
        0x0010:  0a64 c0b5 3f32 0050 ef23 2d5f 0000 0000
        0x0020:  a002 faf0 3d87 0000 0204 05b4 0402 080a
        0x0030:  2443 0db0 0000 0000 0103 0307

[tomastigera]

relates to https://github.com/projectcalico/calico/issues/6544 and https://github.com/projectcalico/calico/issues/4326

[farcaller]

Based on the logs above the dst address seems to be offset 10 bytes and I have no idea where the src address is coming from. Looks like it thinks the packet is a "normal interface" here https://github.com/projectcalico/calico/blob/master/felix/bpf-gpl/skb.h#L79?

[farcaller]

I looked closer into it and pods network actually works as intended (was a different issue). Services are broken, though.