moleculer icon indicating copy to clipboard operation
moleculer copied to clipboard

Redis discoverer implements scan incorrectly for Cluster configurations

Open JordanPawlett opened this issue 4 years ago • 9 comments

https://github.com/moleculerjs/moleculer/blob/6f6225d6079af2116a2b4713c5d00037512131f4/src/registry/discoverers/redis.js#L375

this.client.scanStream is used in multiple locations through the custom redis discoverer. scanStream does not exist on client for Redis.Cluster. I believe must iterate over client.nodes('master') to perform a scan, similar to how the Redis' cacher is written.

JordanPawlett avatar Jun 09 '21 10:06 JordanPawlett

Thanks.

icebob avatar Jun 09 '21 18:06 icebob

Could you create a PR?

icebob avatar Sep 20 '21 17:09 icebob

Now I will make and add tests

intech avatar Sep 21 '21 10:09 intech

The preliminary result is that simply sending the request through master will not work. I'm figuring out if this problem is related to my tests or it's a bug in ioredis

intech avatar Sep 21 '21 22:09 intech

@JordanPawlett

  1. You could clarify how exactly your cluster is assembled through sentinel or redis cluster
  2. Its configuration is how many masters and slaves per master.
  3. And is there a helm or docker-compose with sh scripts to configure it? Or maybe it's some kind of DBaaS solution?

I am testing on 2 different cluster configurations and am getting 2 completely different problems.

intech avatar Sep 23 '21 12:09 intech

I've since moved to use etcd for service-discovery. If i remember correctly the Redis discovery worked correctly with a Sentinel setup. However, breaks for redisCluster, as scanStream does not exist on that client.

Both redis-cluster and redis-sentinel were deployed in a k8s cluster using bitnami helm charts.

What problems specifically are you encountering?

JordanPawlett avatar Sep 23 '21 16:09 JordanPawlett

If i remember correctly the Redis discovery worked correctly with a Sentinel setup. However, breaks for redisCluster, as scanStream does not exist on that client.

Sentinel works differently, unlike redisCluster, a similar problem does not occur there. With redisCluster, additional commands are sent to redis-server which results in an error.

scanStream exists in ioredis and scan command is supported in redis-server with cluster. Errors occur when trying to work with data, the scan command on the server in one case CROSSSLOT error, in another cluster All keys in the pipeline should belong to the same slots allocation group when requesting for any node (master/slave).

intech avatar Sep 23 '21 17:09 intech

Any news about it?

icebob avatar Nov 17 '21 08:11 icebob

Any news about it?

Alas, I don't have a solution yet. In testing, I didn't a solution with client.nodes ('master'). I will run tests again with the current versions of redis and ioredis to make a final conclusion where the problem is.

intech avatar Nov 17 '21 11:11 intech