predixy icon indicating copy to clipboard operation
predixy copied to clipboard

May require additional node.js client query restructuring for queries against predixy

Open daluu opened this issue 5 years ago • 0 comments

I'll add more info & perhaps reproducible test scenario later when I have some time, as we're using a customized fork of predixy for our specific deployment. But just wanted to note this for potential node.js users of redis who may use predixy, to be aware of a possible issue.

I noticed that when switching from regular redis (non-cluster, standalone server) to predixy, using https://github.com/NodeRedis/node_redis, I came across issue when you nest queries that also involve iteration. Like a dependent hgetall that follows an smembers query, where you iterate over the smembers returned for hgetall. However, if we follow smembers with a single hgetall only, then things are ok, it's only an issue if we iterate over a set of smembers for hgetall, whatever N might be (I haven't debugged to find the breaking N).

The particular issue observed for the problem scenario is that smembers may return null (i.e. empty) when we don't expect it to (say when running smember query by itself), or the whole query chain gets stuck and times out eventually. The same query pattern has no issue under regular standalone redis.

An example of the type of query causing the problem can be found in the original sample code from: https://chrysohous.wordpress.com/2012/09/06/redis-and-nodejs-problem-with-asynchronism, including the initial solution. I haven't tried but the updated solution in that post (resulting from a comment from me) might solve this problem. I worked around the issue with a different solution for my use case, avoiding the nested/chained query.

On a related note, had no problems with python redis client migrating from standalone redis to predixy. It seems to be something with the node.js client driver, or the asynchronous nature of node.js in querying redis/predixy, since python client is synchronous I believe.

daluu avatar Oct 30 '18 01:10 daluu