Linux Port Selection Logic Aggressively Shares Ports

[nibanks]

Describe the bug

It seems that the Linux port selection logic will (too) aggressively share local ports between both connected and unconnected sockets. The following happened in a recent test (causing a failure):

quic.log

1. Server (unbound) socket is started with no specified local port. Linux stack assigns port X as local port.
2. Client (unbound) socket is created and then connected to port X. Linux stack also assigns port X as the local port.
3. Client socket sends packet to Server.
4. Client socket receives packet.

 [0][3846.3846][07:15:39.400854][data][0x55ff8e028000] Created, local=[::]:0, remote=None
 [0][3846.3846][07:15:39.401113][list][0x55ff8e026860] Started, Binding=0x55ff8e024a50, LocalAddr=*:41012, ALPN=MsQuicTest

 [1][3846.384d][07:15:39.401545][data][0x7fe56c001e30] Created, local=None, remote=127.0.0.1:41012
 [1][3846.384d][07:15:39.401642][conn][0x55ff8e0506a0] New Local IP: 127.0.0.1:41012

 [1][3846.384d][07:15:39.401925][data][0x7fe56c001e30] Send 1220 bytes in 1 buffers (segment=1220) Dst=127.0.0.1:41012, Src=127.0.0.1:41012
 [1][3846.384c][07:15:39.402035][data][0x7fe56c001e30] Recv 1220 bytes (segment=1220) Src=127.0.0.1:41012 Dst=127.0.0.1:41012
 [1][3846.384c][07:15:39.402039][bind][0x7fe56c001470] DROP packet Dst=127.0.0.1:41012 Src=127.0.0.1:41012 Reason=No listeners registered to accept new connection..

Somehow we need to detect and prevent this from happening.

Affected OS

• [ ] All
• [ ] Windows Server 2022
• [ ] Windows 11
• [ ] Windows Insider Preview (specify affected build below)
• [X] Ubuntu
• [ ] Debian
• [ ] Other (specify below)

MsQuic version

main / ALL

Steps taken to reproduce bug

1. Run test on Linux

Expected behavior

The tests should pass

Actual outcome

Occasionally the tests fail because of shared ports assigned by the Linux stack.

[nibanks]

A bit more on this: Normally, the client doesn't set SO_REUSEPORT on the socket unless UDP port sharing has been explicitly enabled on the client. And this test case is enabling it when running with multiple parallel client connections:

    if (ClientStats->ConnectionCount > 1) {
        Connection->SetShareUdpBinding(true);
    }

I'm not sure we need to be doing this, so we might just disable this code. That being said, I still don't like this behavior, and I think we should add logic to protect from this happening.

[thhous-msft]

This was actually brought up as a known issue when the Share UDP Binding code was added. This is as the linux stack is designed, so fixing the test is the only actionable bug here. From a random program, we're not going to be able to tell if the port grabbed was already given to another port, since that other port might be a part of another program. We could detect if this case occurs in the same program however.