xdp-for-windows
xdp-for-windows copied to clipboard
microsoft
Custom QUIC RSS Support
QUIC would benefit from the ability to customize the RSS logic that dictates which receive queue a packet should be put in, based on the QUIC connection ID, instead of the UDP tuple. If the HW supports the offload, eventually, we'd like to be able to plumb this configuration all the way down. If not (and the HW just does tuple RSS) we still want XDP to put the packet in the proper app queue, even if it doesn't align with HW RSS.
On the send path, we should make sure the app can send on whatever CPU it's executing, inline, all the way to the HW, even if it doesn't match HW RSS.
This is already supported today via the XDP_MATCH_QUIC_FLOW_DST_CID (and SRC_CID) rule, as long as the CID is chosen such that the app ID and target worker thread ID are bitwise-adjacent within the CID. For example, you could set some portion of the CID space to be AAAATTTT where A is an app ID bit and T is a thread ID bit, and then plumb a match rule 00010010 for App1 Thread2 on each receive RSS queue.
Good point, let me think on this a bit more. What do we do if we receive a T (for the initial random CID) that doesn't necessarily match a rule?
Ah, yeah, the initial connection case is trickier, and perhaps solving that problem still generalizes to solving the server-assigned CID problem. We could instead provide a set of sockets to distribute load based on the CID space, which would require a new rule.
As usual, eBPF would allow a lot more flexibility here (by delegating this problem back to the app) but if we were to add a rule that hashes the T bits after steering based on the A bits, you would get consistent delivery for established connections, and fair distribution of new connections if the client randomizes its initial T bits. Of course, part of the ask here is for QUIC to be able to choose the target worker, so the hashing function would need to be at least locally predictable.
I think it's ok to say initials all go to 1 queue, after which QUIC will dynamically distribute (based on calculating custom CIDs) each connection to whatever queue it wants. So the initial CID rule all maps to 1 queue. The post-handshake CIDs are distributed.
Yeah, since this scenario is by definition non-adversarial, and currently focuses on long-lived connections, that should be fine.