Hangfire icon indicating copy to clipboard operation
Hangfire copied to clipboard

Server Shutdown Message?

Open nathvi opened this issue 6 years ago • 6 comments

Is there any sort of server shutdown message where the HangFire server processes its existing queue but stops accepting any new requests?

nathvi avatar Feb 05 '19 14:02 nathvi

In version 1.7.0 there's the BackgroundJobServerOptions.StopTimeout property that sets the TimeSpan value (by default it's zero) for how long to wait for background jobs to be completed without firing CancellationToken. So shutdown pipeline looks like this (they are the log messages):

Before these signals our server is running, and we are able to pick up the new background jobs.

Server rd0003ff2199ca:13116:03c6de91 caught stopping signal...

Starting from this signal above, we don't pick any new jobs, but allow fetched ones to be completed. However, neither IJobCancellationToken nor CancellationToken are triggered.

Server rd0003ff2199ca:13116:03c6de91 caught stopped signal...

Starting from the message above we are triggering cancellation tokens to tell background jobs our server is shutting down, but give them some time to acknowledge that shutdown is triggered and safely abort themselves.

Server rd0003ff2199ca:13116:03c6de91 caught shutdown signal...

After this signal we don't wait for background jobs completion and shutting down the server and calling the RemoveServer method to un-register it.

odinserj avatar Apr 17 '19 11:04 odinserj

Starting from this signal above, we don't pick any new jobs, but allow fetched ones to be completed.

I'm assuming that if there are no pending jobs, then the timeout occurs immediately, instead of always waiting until StopTimeout?

VoidMonk avatar Apr 18 '19 03:04 VoidMonk

Yes, processing server is waiting for completion of all of its background processes, and in this case background processes will be completed immediately, leading to almost immediate completion of a processing server itself.

odinserj avatar Apr 22 '19 08:04 odinserj

Why is the server shutting down? What is the recommended value for StopTimeout? I have a job running when this Server t1:6128:9a9fcab5 caught stopping signal... occurred, and the job hang. in console mode Control-C restarted it, and as a service I had to stop and restart the service.

govininp avatar Jul 10 '19 00:07 govininp

Is there a way that I can configure the 3 timeouts?

A snippet to stoptimeout, and to the cancellationToken timeout and then to the shutdown timeout?

To make it clear, I addedd this code

            services.AddHangfireServer(opts=>
            {
                opts.StopTimeout = TimeSpan.FromSeconds(0);
                opts.CancellationCheckInterval = TimeSpan.FromSeconds(2);
                opts.ShutdownTimeout = TimeSpan.FromSeconds(0);
            });

does this mean that when IIS recycle the hangfire, in 0 seconds it will trigger the cancellation token? after 2 seconds if not all the jobs are cancelled , it will trigger the shutdown and then it will terminate the remaining jobs? Is that @odinserj ?

luizfbicalho avatar Dec 06 '23 19:12 luizfbicalho

any news on this issue @odinserj

luizfbicalho avatar Jan 12 '24 13:01 luizfbicalho