CI build fails due to Azure package feed error 503 (Service Unavailable)
Restore failure seen in https://runfo.azurewebsites.net/view/build/?number=1451898.
Example log:
C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.RestoreEx.targets(19,5): error : (NETCORE_ENGINEERING_TELEMETRY=Restore) Failed to retrieve information about 'Microsoft.VisualStudio.GraphModel' from remote source 'https://pkgs.dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_packaging/825db618-e3eb-4426-ba54-b1d6e6c944d8/nuget/v3/flat2/microsoft.visualstudio.graphmodel/index.json'. at NuGet.Protocol.HttpFileSystemBasedFindPackageByIdResource.<FindPackagesByIdAsync>d__17.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Protocol.HttpFileSystemBasedFindPackageByIdResource.<EnsurePackagesAsync>d__16.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Protocol.HttpFileSystemBasedFindPackageByIdResource.<DoesPackageExistAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.SourceRepositoryDependencyProvider.<FindLibraryCoreAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.SourceRepositoryDependencyProvider.<>c__DisplayClass19_0.<<FindLibraryAsync>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.SourceRepositoryDependencyProvider.<FindLibraryAsync>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.ResolverUtility.<>c__DisplayClass9_1.<<FindLibraryFromSourcesAsync>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at NuGet.DependencyResolver.ResolverUtility.<FindLibraryFromSourcesAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.ResolverUtility.<FindLibraryByVersionAsync>d__8.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.ResolverUtility.<FindPackageLibraryMatchAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.ResolverUtility.<FindLibraryMatchAsync>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at NuGet.DependencyResolver.ResolverUtility.<FindLibraryEntryAsync>d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<CreateGraphNode>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.DependencyResolver.RemoteDependencyWalker.<WalkAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.ProjectRestoreCommand.<WalkDependenciesAsync>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.ProjectRestoreCommand.<TryRestoreAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreCommand.<ExecuteRestoreAsync>d__57.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreCommand.<ExecuteAsync>d__40.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreRunner.<ExecuteAsync>d__7.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreRunner.<ExecuteAndCommitAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreRunner.<CompleteTaskAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at NuGet.Commands.RestoreRunner.<RunAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Commands.RestoreRunner.<RunAsync>d__0.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Build.Tasks.BuildTasksUtility.<RestoreAsync>d__12.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NuGet.Build.Tasks.Console.MSBuildStaticGraphRestore.<RestoreAsync>d__14.MoveNext()
Runfo Tracking Issue: Microsoft.VisualStudio.GraphModel restore failure
| Definition | Build | Kind | Job Name |
|---|---|---|---|
| roslyn-CI | 1921900 | PR 62924 | Build_Windows_Release |
| roslyn-CI | 1921900 | PR 62924 | Build_Windows_Debug |
| roslyn-CI | 1921900 | PR 62924 | Correctness_Rebuild |
| roslyn-CI | 1921900 | PR 62924 | Correctness_Build |
| roslyn-CI | 1921900 | PR 62924 | Correctness_Analyzers |
| roslyn-CI | 1921900 | PR 62924 | Correctness_Rebuild |
| roslyn-CI | 1921900 | PR 62924 | Build_Windows_Debug |
| roslyn-CI | 1921180 | PR 62924 | Build_Windows_Release |
| roslyn-CI | 1921180 | PR 62924 | Build_Windows_Debug |
| roslyn-CI | 1921180 | PR 62924 | Correctness_Rebuild |
| roslyn-CI | 1921180 | PR 62924 | Correctness_Build |
| roslyn-CI | 1921180 | PR 62924 | Correctness_Analyzers |
| roslyn-CI | 1921180 | PR 62924 | Correctness_Rebuild |
| roslyn-CI | 1921180 | PR 62924 | Build_Windows_Debug |
Build Result Summary
| Day Hit Count | Week Hit Count | Month Hit Count |
|---|---|---|
| 0 | 0 | 2 |
@allisonchou the Runfo tracking issue here is too specific. It should be updated to instead match this line in the restore logs:
Response status code does not indicate success: 503 (Service Unavailable)
This no longer seems to be a high hitter.