xdp-forward: VLAN support in flowtable mode

[sbraz]

Hi, I am attempting to use xdp-forward's flowtable mode to improve performance with a 4-port router (2 ports use the igc driver and 2 ports use atlantic). My setup is the following:

• the lan interface is a bridge containing 3 physical interfaces: lan0, lan1 and lan2.
• the wan interface is a VLAN subinterface of the physical wan0 interface (that's a requirement from my ISP).

My setup is that of a simple customer router. Forwarding happens between the lan and wan interfaces. To attempt to improve performance, I have the following nftables configuration:

table inet filter {
    flowtable f {
        hook ingress priority filter
        devices = { lan0, lan1, lan2, wan0 }
    }
    chain forward {
        type filter hook forward priority filter; policy accept;
        meta l4proto { tcp, udp } ct state established,related flow add @f comment "offload established connections"
    }
}

I assume the flowtable works as expected because the rx and tx counters for wan and lan do not seem to follow that of the physical interfaces, and connections are marked with [OFFLOAD] in /proc/net/nf_conntrack.

Now, after setting up the flowtable, I can successfully execute xdp-forward load -f flowtable lan0 lan1 lan2 wan0. I am doing this with kernel 6.12.12-amd64 (Debian Trixie) and xdp-tools 1.5.2.

However, when I check e.g. ethtool -S lan0 or ethtool -S wan0 (both interfaces use the atlantic driver), I see something like this, the redirect counter always stays at 0, only the pass one increases:

     Queue[0] InPackets: 658016
     Queue[0] XdpPass: 18229
     Queue[0] XdpRedirect: 0
     Queue[0] OutPackets: 5028

My understanding is that the XDP program never redirects packets to another interface and I suspect that this happens because forwarding is supposed to occur between a physical interface ( lan0, lan1 or lan2, all part of the lan bridge) and wan, the VLAN interface using the physical interface wan0.

Is there any way to make xdp-forward work in my situation? My tests show that simply adding the Netfilter flowtable doesn't really improve performance (100% CPU usage on one core and lots of dropped packets on wan0 with a single TCP connection at high bit rates) and I was hoping that XDP would help.

[tohojo]

Hmm, not easily, unfortunately. The kernel helpers that xdp-forward uses to figure out where to send the packets don't understand VLANs, so we can't really resolve the packets to the right VLAN devices and make the kernel understand them.

It doesn't look like it's super complicated to add this support to the helpers, actually, so I will make a note to look into that; but this will need to go into the kernel first, before we can support it in xdp-forward.

[sbraz]

Thanks for the explanation.

but this will need to go into the kernel first

Is that something you would implement yourself as well? I'd be happy to test a patched kernel if that helps :)

[tohojo]

but this will need to go into the kernel first

Is that something you would implement yourself as well? I'd be happy to test a patched kernel if that helps :)

Yeah, that's the plan. Certainly not until after I get back from Netdevconf next week, though, and no promises for a timeline after; but will let you know when/if there's something to test, thanks for offering! :)