Hangfire icon indicating copy to clipboard operation
Hangfire copied to clipboard

Add UniqueEnqueue methods

Open kogratte opened this issue 5 years ago • 3 comments

Allow users to enqueue a job replacing an existing one.

Usage:

Background.UniqueEnqueue<RegisterDeviceFunction>(x => x.ExecuteAsync( new RegisterPlateNumberOfDeviceFunctionInput { DeviceId = request.DeviceId, LastConnection = DateTime.Now, PlateNumber = request.PlateNumber } ), args => { if (args[0] is RegisterPlateNumberOfDeviceFunctionInput input) { return input.DeviceId == request.DeviceId; } return false; } );

All scheduled and enqueued job who match the comparison are destroyed before the new one is enqueued.

kogratte avatar May 17 '19 10:05 kogratte

+1

Tested and approved.

Odonno avatar May 17 '19 17:05 Odonno

Thanks for your work, @kogratte, but I can't accept this PR due to the following problems. Consider the following concurrent schedule, where two threads simultaneously call the UniqueEnqueue method with the same comparer:

Step Thread 1 Thread 2
1 FilterExistingJobs<T>(comparer);  
2   FilterExistingJobs<T>(comparer);
3 return Enqueue<T>(methodCall);  
4   return Enqueue<T>(methodCall);

As a result, Thread 2 will not see that Thread 1 is creating the same background job (and vice versa), because background job still wasn't created, and as a result of this race condition, we'll have non-unique behavior. So we simply can't call this method UniqueEnqueue.

Another problem is related with the filtering itself – there may be thousands or millions of background jobs at some point of time. By querying all of them we'll kill our storage.

odinserj avatar May 22 '19 11:05 odinserj

Thanks for the feedback!

One step a time :)

First of all, is there a way to get the job queue that is thread safe? I imagine another option based on attribute usage. Other jobs would be removed on runtime, but there is still the problematic with multiple threads, or even with a big queue.

I think I'm not the first one to meet this need, so, if I'm ready to make it come true, do you see a way to do it without any of the trouble you've identified?

kogratte avatar May 22 '19 12:05 kogratte