aspire icon indicating copy to clipboard operation
aspire copied to clipboard
verified publisher icon dotnet

Odd errors if `Aspire.Dashboard.exe` gets orphaned from a previous run

Open afscrome opened this issue 1 year ago • 5 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Describe the bug

Not entirely sure how I've managed this, but I somehow ended up with an orphaned Aspire.Dashboard.exe process. Once this process was orphaned, all further attempts to start my aspire projects failed with the below logs.

fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.657Z Unhandled exception. System.IO.IOException: Failed to bind to address https://127.0.0.1:17054: address already in use.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z  ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z  ---> System.Net.Sockets.SocketException (10048): Only one usage of each socket address (protocol/network address/port) is normally permitted.
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at System.Net.Sockets.Socket.Bind(EndPoint localEP)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    --- End of inner exception stack trace ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z --- End of stack trace from previous location ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    --- End of inner exception stack trace ---
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Aspire.Dashboard.DashboardWebApplication.Run() in /_/src/Aspire.Dashboard/DashboardWebApplication.cs:line 777
fail: Aspire.Hosting.Dashboard[0]
      2024-09-04T22:44:11.661Z    at Program.<Main>$(String[] args) in /_/src/Aspire.Dashboard/Program.cs:line 7
fail: Aspire.Hosting.Dashboard.Microsoft.Extensions.Hosting.Internal.Host[11]
      Hosting failed to start
      System.IO.IOException: Failed to bind to address https://127.0.0.1:17054: address already in use.
       ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
       ---> System.Net.Sockets.SocketException (10048): Only one usage of each socket address (protocol/network address/port) is normally permitted.
         at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
         at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
         at System.Net.Sockets.Socket.Bind(EndPoint localEP)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
         at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)

Expected Behavior

  1. I'd expect aspire to be able to recover from an orphaned dashboard process - ideally by not orphaning the process in the first place, but if not by killing it and re-starting.
  2. I also don't expect the stack trace above to be duplicated, with each line of the stack trace being it's own error.

Steps To Reproduce

End up with an orphaned Aspire.Dashboard.exe from one aspire run

Exceptions (if any)

No response

.NET Version info

.NET SDK:
 Version:           8.0.401
 Commit:            811edcc344
 Workload version:  8.0.400-manifests.57f7c351
 MSBuild version:   17.11.4+37eb419ad

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.401\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [aspire]
   Installation Source: VS 17.10.35201.131
   Manifest Version:    8.2.0/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.0\WorkloadManifest.json
   Install Type:        FileBased


Host:
  Version:      8.0.8
  Architecture: x64
  Commit:       08338fcaa5

.NET SDKs installed:
  8.0.304 [C:\Program Files\dotnet\sdk]
  8.0.401 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Anything else?

No response

afscrome avatar Sep 04 '24 22:09 afscrome

Yes this is known. Are you using 8.2?

cc @karolz-ms

davidfowl avatar Sep 05 '24 05:09 davidfowl

Yup, I started seeing this after upgrading to 8.2.

afscrome avatar Sep 05 '24 07:09 afscrome

This is an old bug, we just have much better logs now we can pinpoint the issue. I agree it would be ideal if we could not orphan the dashboard (we're not sure why this happens), and if we could kill the previous one if it was still running, that would require keeping state around somewhere...

davidfowl avatar Sep 05 '24 07:09 davidfowl

This is surprising; we made some extra changes to ensure nothing gets orphaned by accident in 8.2. @afscrome can you reproduce consistently?

karolz-ms avatar Sep 06 '24 00:09 karolz-ms

Sorry @karolz-ms haven't seen this much recently, but I've also been pulled away from aspire work for a bit. Feel free to close this and I can re-open if I encounter it again.

afscrome avatar Oct 04 '24 17:10 afscrome

Can confirm this is still happening in Aspire 8.2.1 (seen on amd64 & arm64) /cc @karolz-ms We cannot repro consistently. It just eventually happens after a sufficient amount of stop/start debugging sessions.

oising avatar Oct 08 '24 18:10 oising

Having the same problem, but it feels pretty damn consistent. I have to clean/rebuild between all successful runs before it kills the dashboard. Every second attempt to start AppHost up fails with a port in use error for the dashboard.

Moerup avatar Oct 18 '24 07:10 Moerup

What version of aspire are you using ?

davidfowl avatar Oct 18 '24 08:10 davidfowl

Did not happen with 8.2.1, upgraded to 9 rc1 then it started happening, tried the nightly builds, same thing. Implementing IDisposable on the AppHost fixed it for me.

Moerup avatar Oct 18 '24 18:10 Moerup

Are you using AddNpmApp?

davidfowl avatar Oct 18 '24 20:10 davidfowl

I'm using a few dapr sidecars and components, redis, sqlserver and azure storage emulator and 3 web apis + Blazor Web App.

Moerup avatar Oct 18 '24 22:10 Moerup

This error exists still but I'm going to close this as a dupe of 6704 since a ton of work went into making this more reliable in 9.1

davidfowl avatar Jan 14 '25 07:01 davidfowl