Add support for restarting services from the dashboard

[davidfowl]

In building apps with the aspire, I'm often using the dashboard as a debugging aid, when a process dies it pretty disruptive to have to restart the entire orchestration project in order to reproduce an issue. It would be great to have the ability to restart a process from the dashboard.

[danmoseley]

@karolz-ms can you evaluate whether this is doable for preview? not critical for preview but is a signficant pain point.

[karolz-ms]

I believe the change can be done solely in dashboard view model code. Namely, to restart a service, the model code should:

1. Fetch the corresponding Executable from DCP
2. Memorize the Executable.Spec and annotations.
3. Delete the Executable
4. Re-create the Executable with the same name, Spec and annotations.

I cannot think of any change on the DCP side that would be required for the above to work as desired, but if we discover something missing, by all means open an issue on the DCP repo.

The same technique should also work for restarting Containers.

[dbreshears]

Cleared milestone for dashboard triage to see if it fits

[smitpatel]

@karolz-ms - Do we need to do anything wrt service/endpoint when deleting/re-creating the service? Would the same work for container/executable?

[karolz-ms]

@smitpatel no we don't but we need to make sure that annotations are preserved on the Executable/ExecutableReplicaSet. And the same procedure should also work for Containers. I am going to update my previous comment with this information.

[HClausing]

In building apps with the aspire, I'm often using the dashboard as a debugging aid, when a process dies it pretty disruptive to have to restart the entire orchestration project in order to reproduce an issue. It would be great to have the ability to restart a process from the dashboard.

Just an idea for the future: On some scenarios this restart action can require an authorized user, and optionally with roles or policies requirements.

[aL3891]

Was looking to add a request for this as well, both in the case where a service has died (obv not due to an error on my part) and when ive made a change in the running app and need to restart it. i guess optionally it could also monitor the source for a service and restart it? (assuming hot reload doesnt just work as is)

[davidfowl]

Using dotnet watch handles the latter case on command line and VS does the same when you use ctrl+f5. You can also hot reload when you hit F5

[aL3891]

So dotnet watch is transitive? that's nice! i tried it now though and got errors like this in the logs

dotnet watch 🚀 Started
C:\Program Files\dotnet\sdk\8.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.GenerateAssemblyInfo.targets(199,5): error MSB3713: The file "obj\Debug\net8.0\LoginserverDb.AssemblyInfo.cs" could not be created. The process cannot access the file 'C:\Funrock\mma_backend2\LoginserverDb\obj\Debug\net8.0\LoginserverDb.AssemblyInfo.cs' because it is being used by another process. [C:\Funrock\mma_backend2\LoginserverDb\LoginserverDb.csproj]

and 

CSC : error CS2012: Cannot open 'C:\Funrock\mma_backend2\MMA.ServiceDefaults\obj\Debug\net8.0\MMA.ServiceDefaults.dll' for writing -- 'The process cannot access the file 'C:\Funrock\mma_backend2\MMA.ServiceDefaults\obj\Debug\net8.0\MMA.ServiceDefaults.dll' because it is being used by another process.' [C:\Funrock\mma_backend2\MMA.ServiceDefaults\MMA.ServiceDefaults.csproj]
C:\Program Files\dotnet\sdk\8.0.100\Microsoft.Common.CurrentVersion.targets(4734,5): warning MSB3026: Could not copy "obj\Debug\net8.0\Common.dll" to "bin\Debug\net8.0\Common.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'C:\Funrock\mma_backend2\Common\obj\Debug\net8.0\Common.dll' because it is being used by another process.  [C:\Funrock\mma_backend2\Common\Common.csproj::TargetFramework=net8.0]

Starting normaly without watch works for the same solution (This is from the services themselves, the app host starts normally)

[jooooel]

I haven't had the chance to play around with Aspire yet, but this would be great! It's also part of the main big thing I think Tye was missing: being able to start/stop/restart services on demand: https://github.com/dotnet/tye/issues/876

dotnet watch is nice but has two drawbacks as I see it

1. In large solutions, it's much faster to restart things if there are changes in core libraries that many projects depend on
2. I rarely want to start our entire set of applications, but just a core set. Then, depending on what I'm working on I often want to start up another service (projects, containers, etc) as well.

It would be great if there was a way of communicating with the host via a CLI/API to do these things (kind of like what you can do with docker compose, which is what I'm mainly using now). That would also imply that the host only defines the configuration, and that you can toggle things on and off during the lifetime of the host.

And by the way, I'm really happy that Tye got a revival in Aspire! It has so much potential for the inner dev loop, so I'm glad it did 😊

[davidfowl]

It would be great to get this in for the next preview based on your prototype @tlmii

[aL3891]

Perhaps that is what you've already done but one option would be to be able to start/restart services in watch mode from the dashboard? Maybe that could also be configured in the startup profile? So i can have a profile called "debug flimflam" that is set to Start service A normally, watch service B, ignore service C

[DamianEdwards]

@aL3891 that sounds like #303

[davidfowl]

@drewnoakes This is something we want and I know you were looking at adding command support. This would be the first one.

[andi0b]

@davidfowl Great idea, we really need this!

Another probably much harder to implement idea: An option to keep the orchestrated services running when the app host stops and only apply changes on a restart. Exactly like docker compose or kubernetes do it.

This might be quite tricky for executables (not killing child processes, and re-attach stdout/stderr, keeping state of the previous run), but for containers it should be rather simple.

[davidfowl]

Another probably much harder to implement idea: An option to keep the orchestrated services running when the app host stops and only apply changes on a restart. Exactly like docker compose or kubernetes do it.

This might be quite tricky for executables (not killing child processes, and re-attach stdout/stderr, keeping state of the previous run), but for containers it should be rather simple.

I love the idea but it's out of scope. Feel free to file a new issue.

[thangchung]

+1 That would be handy if we can be able to re-start some services (failed to run), instead of re-run them (some solutions with many projects are quite slow to start)

[danbopes]

I'd also like to see this added, for the simple matter of when an application gets updated (And doesn't support hot reloading), I'd like to be able to restart IT and not the entire cluster.

[davidfowl]

Unfortunately restart isn't enough for that since the apphost won’t rebuild your code. If you’re in visual studio, you have to run without debugging to get a more incremental experience without hot reload.

[MadL1me]

@davidfowl

I think not only restart, but replica count change from dashboard would be very helpful.

For example, we can start with ReplicaCount = 3 for service, but we can change it to "0" and "4", for example. This will insanely help with testing scenarios of failovers locally - test what happens if you just turn off the database, and than rescale it

In my head, this would like smth like this:

This will also resolve issue https://github.com/dotnet/aspire/issues/3095 - as one of the ways to perform "complete stop" of the service (or even restart, after some time, with specified replica count we want)

However, It seems easy to add this feature for .NET apps, but it's might be not so trivial for applications where nodes are required to be in state of consensus (any leader-election, multimaster or master-replica architecture) (for example Kafka or Postgres), because it might be required to add info to all brokers/servers about new added host.

UPD: Decided to move disscussion about replicas to dedicated issue

[davidfowl]

I saw the issue you filed, but restart is what we’ll do first as it’s well understood and useful scenario.

[kvenkatrajan]

FYI - @drewnoakes / @karolz-ms

[davidfowl]

Fixed in https://github.com/dotnet/aspire/issues/295