Worker stops working if same singleton key exists in retry & created state an batch size > 1
Steps to repro
- You should have a stately queue
- You should have
batch_size > 1 - 2 jobs with the same singleton key, one in
createdand another one inretrystates
In the above case the fetch query here will fetch both jobs and attempt to mark them as active. This causes the db to fail with UniqueKeyViolationError since both jobs have the same singleton key. In this situation the worker won't fetch messages forever
I added a comment to the PR.
I've encountered an issue similar to this, with a queue with policy singleton and batchSize > 1. Two jobs sent in quick succession end up stuck in the created state (either with an explicit singletonKey or without).
I will try and address this before v11 goes out. I just fixed the issue regarding different keys, but I haven't yet addressed the issue of batches with the same key
I just pushed a fix for this. For batch sizes > 1, I added a window function so it will only pull 1 job per singleton key.