KeyError when running task from another queue

[jtrain]

You have multiple queues created. e.g.

queue1 = AtLeastOnceQueue(tasks={'task1': task1})
queue2 = AtLeastOnceQueue(tasks={'task2': task2})

Once woken, workers will work through all the tasks on the main job table.

But this worker's queue may not have a task which matches one in its available task list. You'll get a KeyError here: https://github.com/gavinwahl/django-postgres-queue/blob/master/dpq/queue.py#L23

Instead of raising KeyError, the worker should do something useful. e.g.

• Ignore the task, it obviously was meant for someone else
• Not fetch that task in the first place, since it cannot complete it
• Raise a specific error that could be caught e.g. DpqInvalidTask

I will make a PR with a test case highlighting the problem, and I think option 1 or 2 above is the best, i'll implement one of them.

[gavinwahl]

I think if this happens the most likely cause is a programming error that should be reported immediately and not ignored. For example adding a new task type without restarting your worker. I'd want to get an error as soon as possible.

[jtrain]

My PR has a test case showing the error...

see here: https://github.com/gavinwahl/django-postgres-queue/pull/9/files#diff-0dd428d04a9d4f8d2baa476299f0eb03R12-R24

its isn't as you say. Just two queues working on the same job table - each wanting to work on their own tasks. Your description would fit if every queue needs to work on every task.