Unable to listen on certain ports (QUIC_STATUS_ADDRESS_IN_USE)

[rzikm]

Describe the bug

Originally discovered in .NET: https://github.com/dotnet/runtime/issues/71518 It is not possible to start MsQuic listener on certain port range (50000-50059), even though there is no process already listening on that port (the port is not show in netstat output). Binding a socket to this port works without issues.

A bit of debugging shows that the following WSAIoctl call returns WSAEADDRINUSE https://github.com/microsoft/msquic/blob/792140022cb33e587e2292e6ff73c648fb3306f4/src/platform/datapath_winuser.c#L1776-L1785

Affected OS

• [ ] All
• [x] Windows Server 2022
• [x] Windows 11, version 22H2
• [X] Windows 11, version 21H2
• [x] Windows Insider Preview (specify affected build below)
• [ ] Ubuntu
• [ ] Debian
• [ ] Other (specify below)

Additional OS information

Probably windows only

MsQuic version

main

Steps taken to reproduce bug

Attempt to start MsQuic listener on any port in range 50000-50059

Expected behavior

The listener is started and works as on any other port.

Actual outcome

MsQuicListenerStart returns QUIC_STATUS_ADDRESS_IN_USE

Additional details

I tried to search the net to see what is so special about these ports, and it turns out that there is something called Port Exclusion Ranges apparently affects this:

> netsh int ip show excludedportrange protocol=udp

Protocol udp Port Exclusion Ranges

Start Port    End Port
----------    --------
     50000       50059     *
     55642       55741
     63552       63651

* - Administered port exclusions.

Indeed, the other two ranges are also impossible to listen on using MsQuic

[nibanks]

Yes, if some app reserves ports (port exclusions) then it's by design that your QUIC app cannot use it. I'm not sure what you want MsQuic to do here.

[rzikm]

I can still bind a UDP socket to these ports and receive data normally.

Source code

Receiver (C#):

using System.Threading.Tasks;
using System.Buffers;
using System.Net.Sockets;
using System.Net;
using System.Text;

Socket s = new Socket(AddressFamily.InterNetworkV6, SocketType.Dgram, ProtocolType.Udp);
s.DualMode = true;
s.Bind(new IPEndPoint(IPAddress.IPv6Any, 50009));
System.Console.WriteLine("Socket Bound");

byte[] buffer = new byte[1024];
do
{
    int received = await s.ReceiveAsync(buffer);
    System.Console.WriteLine(Encoding.ASCII.GetString(buffer.AsSpan(0, received)));
} while (true);

Sender (powershell):

$client = [UdpClient]::new()
$client.Send([System.Text.Encoding]::ASCII.GetBytes("hello"), 5, [IPEndpoint]::new([ipaddress]::Loopback, 50009))

Which makes it look like a limitation of MsQuic. If it is not something you can fix (or don't think it is worth it), then we need to mention it in docs.

[nibanks]

MsQuic grabs a port reservation on Windows (to deal with the other bug .NET brought about MsQuic incorrectly sharing ports) to restrict access to just the current process, but still allow for multiple per-processor sockets to be opened. This is what causes MsQuic to fail to bind to those ports.

[nibanks]

We opened this internal bug to track possibly support this scenario in Windows:

• Bug 40464377: Apps that create port reservations for binding are blocked by port exclusions

[mahi072]

Hi @rzikm, I would like to work on this bug, if it is still open. Please guide me through this. Waiting for your reply.

[rzikm]

Hi @mahi072, unfortunately, this bug can't be solved on .NET side. It is caused by Windows behavior. And would need changes in Windows itself. This issue is open only to track it.