disque icon indicating copy to clipboard operation
disque copied to clipboard

QLEN inconsistent results

Open dolzenko opened this issue 7 years ago • 5 comments

Given the cluster of 3 nodes, shouldn't the QLEN return the same results regardless the node being asked? In my tests only original node returns proper queue length, but GETJOB on different node still works properly

$ redis-cli -h eu1-queue-3 -p 7712 addjob testqueue testjob 1000
D-54bdf6f9-2qZjI4HgYKs8vP0ZWSeAANbK-05a1

$ redis-cli -h eu1-queue-1 -p 7712 qlen testqueue
(integer) 0 # WHY?
$ redis-cli -h eu1-queue-2 -p 7712 qlen testqueue
(integer) 0
$ redis-cli -h eu1-queue-3 -p 7712 qlen testqueue
(integer) 1

$ redis-cli -h eu1-queue-1 -p 7712 getjob from testqueue
1) 1) "testqueue"
   2) "D-54bdf6f9-2qZjI4HgYKs8vP0ZWSeAANbK-05a1"
   3) "testjob"

dolzenko avatar Dec 07 '17 13:12 dolzenko

Try this instead $ redis-cli -h eu1-queue-3 -p 7712 addjob testqueue testjob 1000 replicate 3

mathieulongtin avatar Dec 07 '17 16:12 mathieulongtin

@mathieulongtin I could, but isn't the default to replicate to all nodes? I would expect QLEN to be consistent with GETJOB :/

dolzenko avatar Dec 07 '17 16:12 dolzenko

It's not replicated by default. By not specifying replicate, you're asking for one copy. You're saying "I don't need this job to survive a server failure".

mathieulongtin avatar Dec 07 '17 18:12 mathieulongtin

Sorry for annoyance but README claims replication is by default. Also if jobs are not replicated by default does that mean that clients need to find out cluster size and set replicate param accordingly?

Disque is a synchronously replicated job queue. By default when a new job is added, it is replicated to W nodes before the client gets an acknowledgement about the job being added. W-1 nodes can fail and the message will still be delivered.

dolzenko avatar Dec 08 '17 08:12 dolzenko

Ooop, wrong button

dolzenko avatar Dec 08 '17 08:12 dolzenko