Running jobs in a serial queue?

[PaulAik]

Is there any way I can ensure jobs are run serially as opposed to in parallel (even better - have a serial queue)? E.g. I'd like to ensure only one job of type X can be running at any given time.

I've reduced my number of workers down to 1, not sure if that'll do the trick for now? Does the worker count always get used even if I use BackgroundJob.Enqueue?

[maurodx]

I've also this kind of request: I want to be able to deny a job to be queued if I have the same type of job already in queue and not processed

[odinserj]

All feature requests already implemented :smile::

[DisableConcurrentExecution]
public void SomeMethod() { /* ... */ }

[PaulAik]

Great! Does this feature also cover running jobs of different types concurrently? E.g. if I don't want JobType1 running at the same time as JobType2?

[odinserj]

Not yet, but you can look at the implementation and easily build what you want.

[dotnetdan]

Just to confirm, does [DisableConcurrentExecution] guarantee that jobs will be processed in the order they were enqueued? In the event that one of the jobs on the queue fails with retry disabled, it will continue processing the others in order?

[dukefama]

Hi, I am interested in the response to dotnetdan's question. Is there anyone that can assist with this?

[JarrodJ83]

@dotnetdan I am seeing the exact opposite, actually. It seems the last in is the first processed. In below screenshot the "complete" job was the last thing I enqueued but it was the first thing that ran:

I would think this is a bug. Would like to hear @odinserj 's feedback, though.

[RudeySH]

Any update on this? After reading this thread I can't help but feel Hangfire queues are pretty useless. Is it still not possible to stop concurrent execution of different kinds of jobs? And was the queue order bug ever fixed?

[nmurf]

@odinserj, would you please respond to @JarrodJ83?

[BalassaMarton]

This is something we also need: we use HF in an event sourcing scenario where it is required that

• enqueued jobs (of the same method) run in strict FIFO order
• they are not processed in parallel, ever.

[brandlink-james]

This way works for me. app.UseHangfireServer(options: new BackgroundJobServerOptions { ServerName = "SequentialServer", WorkerCount = 1 });

[dazbradbury]

This way works for me. app.UseHangfireServer(options: new BackgroundJobServerOptions { ServerName = "SequentialServer", WorkerCount = 1 });

This is great unless you have multiple hangfire servers. At that point, you need a way to ensure a single worker queue in a distributed environment, across deploys etc. Anyone have an elegant solution to that?

[Sire]

Check out their paid version, it might solve this: https://www.hangfire.io/ace/

[SoftCircuits]

[DisableConcurrentExecution]

Error CS1729 'DisableConcurrentExecutionAttribute' does not contain a constructor that takes 0 arguments

[auvobws]

Did anyone figure out how to get the jobs to stay in enqueued until the first job is completely done: either failed/suceeded and not scheduled?

[DMiradakis]

I don't think this is offered in Hangfire, neither in the free version nor in the paid versions.

As far as I can tell, the "type" of queue used in Hangfire is what I've heard called a Heap Queue. In AWS Simple Queue Service, they call it a Standard Queue. The idea is that jobs are executed in a best-effort ordering in a highly concurrent manner, where exact ordering is NOT guaranteed and instead throughput is prioritized.

It sounds like what many of you are looking for is a FIFO queue, where strict ordering is guaranteed and throughput is deprioritized.

You'll probably need to open a PR and design this yourself, or use another service. I'm building an open source .NET job orchestrator called Didact, and I'll be including both Heap Queues and FIFO Queues. Pretty soon I'll have some docs on this on the docsite.