django-rq
django-rq copied to clipboard
add doc for redis cluster usage if support
Hi, I see rq support redis cluster from https://github.com/rq/rq/pull/741, but django-rq does not provide doc or details for using redis cluster.
I see code from https://github.com/rq/django-rq/blob/v1.0.1/django_rq/queues.py#L65-L107
def get_redis_connection(config, use_strict_redis=False):
"""
Returns a redis connection from a connection config
"""
redis_cls = redis.StrictRedis if use_strict_redis else redis.Redis
if 'URL' in config:
return redis_cls.from_url(config['URL'], db=config.get('DB'))
if 'USE_REDIS_CACHE' in config.keys():
try:
from django.core.cache import caches
cache = caches[config['USE_REDIS_CACHE']]
except ImportError:
from django.core.cache import get_cache
cache = get_cache(config['USE_REDIS_CACHE'])
if hasattr(cache, 'client'):
# We're using django-redis. The cache's `client` attribute
# is a pluggable backend that return its Redis connection as
# its `client`
try:
# To get Redis connection on django-redis >= 3.4.0
# we need to use cache.client.get_client() instead of
# cache.client.client used in older versions
try:
return cache.client.get_client()
except AttributeError:
return cache.client.client
except NotImplementedError:
pass
else:
# We're using django-redis-cache
try:
return cache._client
except AttributeError:
# For django-redis-cache > 0.13.1
return cache.get_master_client()
if 'UNIX_SOCKET_PATH' in config:
return redis_cls(unix_socket_path=config['UNIX_SOCKET_PATH'], db=config['DB'])
return redis_cls(host=config['HOST'], port=config['PORT'], db=config['DB'], password=config.get('PASSWORD', None))
Django-rq checks URL
,USE_REDIS_CACHE
to determine which connection to use. So I use below django settings:
RQ_DB = int(os.getenv('REDIS_DB_NAME', 0))
# RQ_PASSWORD = os.getenv('REDIS_PASSWORD', '')
RQ_TIMEOUT = int(os.getenv('REDIS_TIMEOUT', 360))
REDIS_URL = os.getenv('REDIS_URL','localhost:6379')
RQ_QUEUES = {
'default': {
'URL': 'redis://{}'.format(REDIS_URL),
'DB': RQ_DB,
'DEFAULT_TIMEOUT': RQ_TIMEOUT
},
'high': {
'URL': 'redis://{}'.format(REDIS_URL),
'DB': RQ_DB,
'DEFAULT_TIMEOUT': RQ_TIMEOUT
},
'low': {
'URL': 'redis://{}'.format(REDIS_URL),
'DB': RQ_DB,
'DEFAULT_TIMEOUT': RQ_TIMEOUT
}
}
RQ = {
'DEFAULT_RESULT_TTL': int(os.getenv('RQ_RESULT_TTL', 5000)),
}
REDIS_URL is 10.24.0.7:14020,10.24.0.7:14021,10.24.0.7:14022,10.24.0.7:14023,10.24.0.7:14024,10.24.0.7:14025 which is cluster url. But django-rq raise exception.
Django-rq seems not support redis cluster? So can django-rq support redis cluster or provide detailed doc if it already support.
@adolphlwq I would like to use redis cluster. Have you solved this? Thanks.
@michaelluk No, Have some trouble in pr https://github.com/Grokzen/redis-py-cluster/pull/251 I closed it.
@adolphlwq Ok, thanks. I think I will try to use the redis cluster proxy (https://github.com/eleme/corvus) then.
@adolphlwq Any update here? We want to use it with a cluster, but we think this is still not officially supported, isn't it? Would be great to get any update here.