Test dotnet CDN Performance

[richlander]

We're in the process of changing the Content Delivery Network (CDN) we use, for https://github.com/dotnet/core/issues/9671. CDNs are a major source of performance on the web. In the past, some users have come to us with reports of very poor download performance, particularly in Asia. We want to proactively start a conversation about download performance and to make it easy to perform tests.

Testing links:

• https://builds.dotnet.microsoft.com/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
• https://dotnetcli.azureedge.net/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
• https://download.visualstudio.microsoft.com/download/pr/805cdca8-ac43-4d76-8ce8-efd11f1997f2/17aeb8b0cd34c6f8d80217bf6a4ed3cd/dotnet-runtime-8.0.11-linux-x64.tar.gz
• https://www.nuget.org/api/v2/package/Microsoft.NETCore.App.Runtime.linux-x64/8.0.11
• https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.runtime.linux-x64/8.0.11/microsoft.netcore.app.runtime.linux-x64.8.0.11.nupkg

The first three links are copies of the same file, while the last two are copies of a slightly larger variant of the same content. They all go through different networks paths, which make them good candidates to test network performance.

If you have any concerns about network performance, please test the files and report back if you believe the results are bad. We expect that our CDN architecture will change multiple times during the month of January. It may be useful for concerned users to test more than once and to save logs.

It is best to test the results via the command line although the browser works, too.

Example test

The following example test was run on a home network near Seattle, WA.

$ curl -LO https://builds.dotnet.microsoft.com/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0  26.5M      0  0:00:01  0:00:01 --:--:-- 26.5M
$ curl -LO https://dotnetcli.azureedge.net/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0  72.7M      0 --:--:-- --:--:-- --:--:-- 72.9M
$ curl -LO https://download.visualstudio.microsoft.com/download/pr/805cdca8-ac43-4d76-8ce8-efd11f1997f2/17aeb8b0cd34c6f8d80217bf6a4ed3cd/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0  39.2M      0 --:--:-- --:--:-- --:--:-- 39.2M
$ curl -LO https://www.nuget.org/api/v2/package/Microsoft.NETCore.App.Runtime.linux-x64/8.0.11
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 33.3M  100 33.3M    0     0  35.1M      0 --:--:-- --:--:-- --:--:-- 76.4M
$ curl -LO https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.runtime.linux-x64/8.0.11/microsoft.netcore.app.runtime.linux-x64.8.0.11.nupkg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 33.3M  100 33.3M    0     0  15.6M      0  0:00:02  0:00:02 --:--:-- 15.6M

As you can see, these results are good, downloading the .NET 8 runtime in 1s or less. We hope for similar results for others. We'll also be investigating the speed differences between the CDNs.

[Varorbc]

@richlander I've put together a comparison of download speeds in the China region.

PS D:\Downloads> curl -LO https://builds.dotnet.microsoft.com/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0  5480k      0  0:00:05  0:00:05 --:--:-- 6852k
PS D:\Downloads> curl -LO https://dotnetcli.azureedge.net/dotnet/Runtime/8.0.11/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0   980k      0  0:00:31  0:00:31 --:--:-- 1234k
PS D:\Downloads> curl -LO https://download.visualstudio.microsoft.com/download/pr/805cdca8-ac43-4d76-8ce8-efd11f1997f2/17aeb8b0cd34c6f8d80217bf6a4ed3cd/dotnet-runtime-8.0.11-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.8M  100 29.8M    0     0  2262k      0  0:00:13  0:00:13 --:--:--  858k
PS D:\Downloads> curl -LO https://www.nuget.org/api/v2/package/Microsoft.NETCore.App.Runtime.linux-x64/8.0.11
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   228    0   228    0     0    231      0 --:--:-- --:--:-- --:--:--   231
100 33.3M  100 33.3M    0     0  1031k      0  0:00:33  0:00:33 --:--:--  688k
PS D:\Downloads> curl -LO https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.runtime.linux-x64/8.0.11/microsoft.netcore.app.runtime.linux-x64.8.0.11.nupkg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   266  100   266    0     0    674      0 --:--:-- --:--:-- --:--:--   676
100 33.3M  100 33.3M    0     0  5244k      0  0:00:06  0:00:06 --:--:-- 5867k

[richlander]

Thanks! That info is quite useful. We will plan to switch builds.dotnet.microsoft.com to 100% Akamai next week. It is currently 100% AFD. Later, we'll likely make them 50/50. If you are willing, I'll ask you to test again next week, after we've made that change.

At the time of writing, these domains are, in order:

• 100% AFD
• 100% Edgio
• Unknown

Note: I don't actually know what download.visualstudio.com uses. We'll continue to use it, but are going to use it less, so it's not going to be a primary CDN for us.

[Varorbc]

Alright, just let me know whenever you've finished switching over the traffic. I'm happy to help with testing whenever you're ready.

[Varorbc]

I've got a bit of a question. Why not stick with Microsoft's own services instead of going with another company? Is there a chance we might run into issues like we did with Edgio going bankrupt in the future?

[richlander]

Good question. We are going to use both AFD and Akamai. It is important to use multiple CDNs so that we are protected from service degradation. We won't run into the same problem since we can change CDNs transparently, temporarily or permanently. A custom domain avoids a tie to a specific CDN (like we have had) and traffic manager enables us to make changes w/o needing to touch DNS records.

We learned that NuGet was already doing this. If we'd talked to them about this earlier, we would have been able to avoid all of this. We are now working together to align our approach, which will be better for everyone.

[richlander]

I just realized we can add NuGet download to the list. It's a slightly different file, but close enough to be valuable for this test. I added it to my initial comment. Please test with that as well. My understanding is that the NuGet API endpoint is using traffic manager as well, with a mix of CDNs, some of which are geo-specific. Like us, they are progressively adding in Akamai CDN so the numbers may change over the month before becoming a more stable mix.

[Varorbc]

I've already tested the NuGet downloads.

[richlander]

@joelverhagen gave us another NuGet link to test. I added it above.

[Varorbc]

I added it above.

[richlander]

Thanks much. That's looks much better!

[Varorbc]

We will plan to switch builds.dotnet.microsoft.com to 100% Akamai next week.

I saw a news article this morning about Akamai pulling out of China in 2026. What's your take on this? How does it impact our plans? Are we still going ahead with the switch?

[richlander]

We saw that, too. It doesn't impact anything. We were not relying on Akamai for a solution for China. We are considering adopting the same solution as NuGet has for their api endpoint (which they are still optimizing). It relies on a China-specific CDN operated by another party, which is routed via a geo load balancer.

[Varorbc]

@richlander Why does the domain name ci.dot.net download so slowly?

[richlander]

We have seen this before (in Europe). We need to look at this again. Thanks for the report.