Support for Redis Cluster by providing an alternative to pipelining

[peteAhn]

After I read the Redis cluster connection in the readme document, I've tested the sample source connecting the Redis Cluster with 3 nodes. Then I faced the following error message.

ReplyError: CROSSSLOT Keys in request don't hash to the same slot
return new ReplyError(string)
           ^
ReplyError: CROSSSLOT Keys in request don't hash to the same slot
    at parseError (/home/cr-api-server/node_modules/redis-parser/lib/parser.js:179:12)
    at parseType (/home/cr-api-server/node_modules/redis-parser/lib/parser.js:302:14) {
  command: { name: 'exec', args: [] },
  previousErrors: []
}

I found out the following issues.

• Resque issue #1301
• CROSSSLOT Keys error in Redis Cluster

Please let me know the node-resque support a Redis Cluster or not. Thanks.

[evantahler]

Hi @peteAhn!

Yes, at the moment there are a few places in the code that don't work with Redis Cluster due to pipelining multiple commands together - like this example, as the keys used might be on different nodes. Ideally we do want to sue pipelining as it offers better atomicity.

Redis cluster also likely won't work well with our atomic popAndStore Lua script because again those same keys might be on different nodes. Here's how we currently mitigate this.

Are you able to help us by making a utility method that allows us to use mutli/exec pipelining when available, and falls back to parallel commands when using redis cluster?

In the short term, you can configure the key slots on your redis cluster to guarantee all the keys resque uses are on the same now, and things should work out... but that sort of removes the point of using a cluster to begin with.

[peteAhn]

Hi @peteAhn!

Yes, at the moment there are a few places in the code that don't work with Redis Cluster due to pipelining multiple commands together - like this example, as the keys used might be on different nodes. Ideally we do want to sue pipelining as it offers better atomicity.

Redis cluster also likely won't work well with our atomic popAndStore Lua script because again those same keys might be on different nodes. Here's how we currently mitigate this.

Are you able to help us by making a utility method that allows us to use mutli/exec pipelining when available, and falls back to parallel commands when using redis cluster?

In the short term, you can configure the key slots on your redis cluster to guarantee all the keys resque uses are on the same now, and things should work out... but that sort of removes the point of using a cluster to begin with.

@evantahler

Thanks for your quick response. When available I will help you, but currently I cannot.