adding a custom header with sequence numbers

[sachintiptur]

Hi All,

Is there any xdp api or mechanism which i can use to add a custom header(includes a seq. number and next_proto fields) to packets and pass it to kernel stack. Basic idea is to, add a sequence number to received packet at XDP layer and capture the same pkt at tc hook and clone it, send replicated packets on disjoint paths to destination. On destination side, eliminate duplicate pkt by checking the sequence numbers. Please suggest.

Thanks, Sachin

[tohojo]

sachints123 [email protected] writes:

Hi All,

Is there any xdp api or mechanism which i can use to add a custom header(includes a seq. number and next_proto fields) to packets and pass it to kernel stack.

Basic idea is to, add a sequence number to received packet at XDP layer and capture the same pkt at tc hook and clone it, send replicated packets on disjoint paths to destination. On destination side, eliminate duplicate pkt by checking the sequence numbers.

Please suggest.

You can do whatever you want with the packet contents. You just add/remove space at the front of the packet with bpf_xdp_adjust_head() and write whatever you want into that space.

Passing such a packet to the stack may confuse it, though, so probably better to write the header in the TC hook on egress, and then just read, process and remove the header in XDP before the packet hits the stack.

You can use maps, or the metadata area in each packet, to communicate between XDP and TC programs.

[sachintiptur]

You mean, I can add custom header on TC egress hook on sending host and remove the header on receiver side XDP? In that case I can't use bpf_xdp_adjust_head() right? Is it possible to communicate(using xdp meta data) between XDP and TC hooks which are on different machines? OR xdp meta data can be processed only on local machine?

[tohojo]

sachints123 [email protected] writes:

You mean, I can add custom header on TC egress hook on sending host and remove the header on receiver side XDP?

Yes.

In that case I can't use bpf_xdp_adjust_head() right?

You'd use that for removing it, and bpf_skb_adjust_room() in the TC hook for adding it.

Is it possible to communicate(using xdp meta data) between XDP and TC hooks which are on different machines? OR xdp meta data can be processed only on local machine?

No. BPF is neat, but it's not magic; if you're going to communicate between hosts you're going to have to tack something on to the packet itself...

[sachintiptur]

sachints123 [email protected] writes: You mean, I can add custom header on TC egress hook on sending host and remove the header on receiver side XDP? Yes. In that case I can't use bpf_xdp_adjust_head() right? You'd use that for removing it, and bpf_skb_adjust_room() in the TC hook for adding it. Is it possible to communicate(using xdp meta data) between XDP and TC hooks which are on different machines? OR xdp meta data can be processed only on local machine? No. BPF is neat, but it's not magic; if you're going to communicate between hosts you're going to have to tack something on to the packet itself...

bpf_skb_adjust_room() supports only IPv4 and IPv6 pkts to add or shrink pkt. In my case, I want to add custom header to PROFINET pkt which is not IP based. Any other way of doing this?

[tohojo]

sachints123 [email protected] writes:

bpf_skb_adjust_room() supports only IPv4 and IPv6 pkts to add or shrink pkt. In my case, I want to add custom header to PROFINET pkt which is not IP based. Any other way of doing this?

Ah, no, if there's no IP header at all you may be out of luck on the TX side. That would require a proper XDP egress hook, which we may get eventually but it's not there right now...

[sachintiptur]

Ohh :( ok, Now I am trying, assuming tc ingress hook in step 2 will not harm

1. eth0 (at tc ingress redirect to eth1)
2. eth1 (at xdp add meta and custom header -> tc ingress, check meta data and clone the pkt and redirect to eth2)
3. eth2 - tcpdump and see if new custom header is added.

WILL THIS WORK???

[tohojo]

sachints123 [email protected] writes:

Ohh :( ok, Now I am trying, assuming tc ingress hook in step 2 will not harm

1. eth0 (at tc ingress redirect to eth1)
2. eth1 (at xdp add meta and custom header -> tc ingress, check meta data and clone the pkt and redirect to eth2)
3. eth2 - tcpdump and see if new custom header is added.

WILL THIS WORK???

I don't think so; if the kernel doesn't recognise the protocol field, it'll most likely just drop the packet after XDP_PASS. Any modification you do at the XDP step will be just like if the machine just received such a packet directly from the network...

[sachintiptur]

sachints123 [email protected] writes: Ohh :( ok, Now I am trying, assuming tc ingress hook in step 2 will not harm 1. eth0 (at tc ingress redirect to eth1) 2. eth1 (at xdp add meta and custom header -> tc ingress, check meta data and clone the pkt and redirect to eth2) 3. eth2 - tcpdump and see if new custom header is added. WILL THIS WORK??? I don't think so; if the kernel doesn't recognise the protocol field, it'll most likely just drop the packet after XDP_PASS. Any modification you do at the XDP step will be just like if the machine just received such a packet directly from the network...

I tried and it's working. I am able to do above. I redirected incoming pkts to another interface and used xdp prog to add custom header and passed on to tc ingress hook and cloning pkt there. Thank you Toke Hoiland...your inputs helped a lot.

[tohojo]

That's great! You're welcome :)