resque-lock icon indicating copy to clipboard operation
resque-lock copied to clipboard

Allow jobs to be queuable once the existing job starts performing

Open lazyatom opened this issue 8 years ago • 0 comments

This is useful when you know that the currently running job might not be taking into account the most recent updates to some of the data it is operating on, which can occur when the jobs take some time to run.

For example:

  1. update model
  2. queue job to index in ElasticSearch
  3. job starts performing
  4. another update happens to the model
  5. try to queue job to update index

With the default queue locking mechanism, step 5 will only succeed in enqueuing a new job if the job that started in step 3 has already completed. However, if that job hasn't finished, but has loaded the model with only the changes from step 1, then the changes introduced in step 4 won't get indexed.

With unlock_while_performing set to true, there can always be a single pending job in the queue, even when there's also a job running, and so we can be confident that any un-indexed changes to the model will get indexed once the final job finishes running.

lazyatom avatar Sep 07 '16 21:09 lazyatom