qp icon indicating copy to clipboard operation
qp copied to clipboard

Job data isn't always available when using redis cluster

Open TheDeveloper opened this issue 9 years ago • 1 comments

When using QP with a redis cluster, job data is sometimes undefined when a job is pulled off the queue and given to a worker.

This could be due to job data and job list residing on different redis instances, so that if your consumer is receiving jobs as quickly as possible after they are queued, the job data might not have finished saving to the other redis instance before the job is given to the worker.

A workaround for this is to use key tags around your queue name e.g. {queueName} so that all QP keys are allocated to a single redis node.

TheDeveloper avatar Jan 05 '16 11:01 TheDeveloper

Might be worthwhile waiting for the job data to be successfully saved before queueing the job, rather than fire-and-forget, feel free to pull req it... probably worth making it another option too, so by default it still fires-and-forgets because it's quicker and only doesnt work in cluster mode when one node is being slower

simontabor avatar Jan 05 '16 15:01 simontabor