celery-redis-cluster-backend icon indicating copy to clipboard operation
celery-redis-cluster-backend copied to clipboard

Fixed setex method call : argument position was wrong

Open sunny316 opened this issue 5 years ago • 6 comments

Fixed setex method call : argument position was wrong

sunny316 avatar Oct 16 '19 11:10 sunny316

Fixed setex method call : argument position was wrong

I aslo find this problem. The script[redis_cluster.py] can works with your PR,but i can't use it within celery. I can't understand second step in readme.md, How should I use the configuration file(celeryconfig.py) in celery. can you help me?

malone6 avatar Mar 16 '20 03:03 malone6

Fixed setex method call : argument position was wrong

I aslo find this problem. The script[redis_cluster.py] can works with your PR,but i can't use it within celery. I can't understand second step in readme.md, How should I use the configuration file(celeryconfig.py) in celery. can you help me?

@malone6 The second step is adding the two config settings to the celery object that you create i.e.

celtasks = Celery(app_name, broker=broker_url) celtasks.conf["CELERY_RESULT_BACKEND"] = "celery_redis_cluster_backend.redis_cluster.RedisClusterBackend" celtasks.conf["CELERY_REDIS_CLUSTER_SETTINGS"] = { 'startup_nodes': [ {"host": "localhost", "port": "6379"}, {"host": "localhost", "port": "6380"}, {"host": "localhost", "port": "6381"} ]}

sunny316 avatar Mar 16 '20 07:03 sunny316

Fixed setex method call : argument position was wrong

I aslo find this problem. The script[redis_cluster.py] can works with your PR,but i can't use it within celery. I can't understand second step in readme.md, How should I use the configuration file(celeryconfig.py) in celery. can you help me?

@malone6 The second step is adding the two config settings to the celery object that you create i.e.

celtasks = Celery(app_name, broker=broker_url) celtasks.conf["CELERY_RESULT_BACKEND"] = "celery_redis_cluster_backend.redis_cluster.RedisClusterBackend" celtasks.conf["CELERY_REDIS_CLUSTER_SETTINGS"] = { 'startup_nodes': [ {"host": "localhost", "port": "6379"}, {"host": "localhost", "port": "6380"}, {"host": "localhost", "port": "6381"} ]}

Thank you for your prompt reply. celtasks = Celery(app_name, broker=broker_url) In this line,broker url should be a SQS uri like 'redis://localhost:6379/0' 'amqp://guest:guest@localhost:5672//'.

but I use one client of redis-cluster as the SQS, it was start errorredis.exceptions.ResponseError: MOVED 14663 172.50.0.4:6393when i config redis-single-url like broker='redis://192.168.2.137:6391/0'. How should i solve this problem if i want ensure HA of celery tasks?

malone6 avatar Mar 16 '20 08:03 malone6

I dont think Celery supports Redis Cluster as its broker(It supports redis for sure). It does not support it for its backend for sure, thats the reason we have to use this package. Try with another broker which is supported by celery like RabbitMQ etc.

sunny316 avatar Mar 16 '20 11:03 sunny316

Oh, I see. I confused broker and backend before this. Thank you for your answers and suggestions. I will try something else and that looks like there is still a long way to use redis-cluster as celery-broker.

malone6 avatar Mar 18 '20 07:03 malone6

Oh, I see. I confused broker and backend before this.

@malone6 sorry my dummy question, but what's the difference? I assume (but I'm not sure) that backend is for background tasking and broker is really as a message broker, is it right?

I was looking for an implementation of celery with Redis cluster as message broker and I ended up here, but I guess again it's a dead end.

robertofalk avatar Oct 21 '21 12:10 robertofalk