Configuration option for changed ssl_cert_reqs default behaviour in redis-py

[ghost]

Hello,

we're experiencing connection issues with Amazon AWS ElastiCache Redis instances over TLS (rediss://...). After raising an issue with redis-py, this seems to be related to a changed default setting/behaviour in redis-py 3.x for the argument "ssl_cert_reqs". Related ticket is: https://github.com/andymccurdy/redis-py/issues/1080 It seems this cannot be configured in django-redis? Can you please add a configuration option?

[jplock]

I tried this, but it didn't work:

"OPTIONS": {
  "REDIS_CLIENT_KWARGS": {
    "ssl_cert_reqs": None,
    "ssl": True
  }
}

I would prefer to verify the CA bundle used by Elasticache, but I don't know where to find that.

[michael-k]

This worked for us:

CACHES = {
    "default": {
        # …
        "OPTIONS": {
            "CONNECTION_POOL_KWARGS": {
                "ssl_ca_certs": "/etc/ssl/certs/ca-certificates.crt",
            },
        },
    },
}

[jplock]

@michael-k just to confirm, is that working for you connecting to Elasticache now? Thanks for the insight!

[michael-k]

@jplock Yes it is. We're running our code inside debian-stretch docker images. You might need to adjust the filename (and maybe the path?) if you run your code in a different environment.

/etc/pki/tls/certs/ca-bundle.crt worked here: https://github.com/andymccurdy/redis-py/issues/1080#issuecomment-445058335

On AWS Lambda these files exist, but I don't know which one to pick¹: /etc/ssl/certs/ca-bundle.trust.crt, /etc/ssl/certs/ca-bundle.crt

¹ Update: see https://serverfault.com/questions/620003/difference-between-ca-bundle-crt-and-ca-bundle-trust-crt; we now use /etc/ssl/certs/ca-bundle.trust.crt and it works :)

[tclancy]

For anyone else finding this and getting blocked by unexpected keyword argument 'ssl_ca_certs', you need to change your LOCATION parameter in a very subtle way, from

'LOCATION': 'redis://{}:6379/0'.format(REDIS_ADDRESS),
'LOCATION': 'rediss://{}:6379/0'.format(REDIS_ADDRESS),

The extra S is for SSL!

[kashypAkash]

Hi Everyone,

I'm trying to connect AWS elasticache(cluster mode) with TLS enabled, the library versions and django cache settings as below

====Dependencies======
redis==3.0.0
redis-py-cluster==2.0.0
django-redis==4.11.0

======settings=======
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': "redis://xxxxxxx.mc-redis-cache-v2.zzzzz.usw2.cache.amazonaws.com:6379/0",
        'OPTIONS': {
            'PASSWORD': '<password>',
            'REDIS_CLIENT_CLASS': 'rediscluster.RedisCluster',
            'CONNECTION_POOL_CLASS': 'rediscluster.connection.ClusterConnectionPool',
            'CONNECTION_POOL_KWARGS': {
                'skip_full_coverage_check': True,
                "ssl_cert_reqs": False,
                "ssl": True
            }
        }
    }
}

It doesn't seem to be a problem with client-class since I'm able to access

from rediscluster import RedisCluster
startup_nodes = [{"host": "redis://xxxxxxx.mc-redis-cache-v2.zzzzz.usw2.cache.amazonaws.com", "port": "6379"}]

rc = RedisCluster(startup_nodes=startup_nodes, ssl=True, ssl_cert_reqs=False, decode_responses=True, skip_full_coverage_check=True, password='<password>')

rc.set("foo", "bar")
rc.get('foo')
'bar'

but I'm seeing this error when django service is trying to access the cache, is there any configuration detail that I might be missing?

File "/usr/lib/python3.6/site-packages/django_redis/cache.py", line 32, in _decorator
    return method(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/django_redis/cache.py", line 81, in get
    client=client)
  File "/usr/lib/python3.6/site-packages/django_redis/client/default.py", line 194, in get
    client = self.get_client(write=False)
  File "/usr/lib/python3.6/site-packages/django_redis/client/default.py", line 90, in get_client
    self._clients[index] = self.connect(index)
  File "/usr/lib/python3.6/site-packages/django_redis/client/default.py", line 103, in connect
    return self.connection_factory.connect(self._server[index])
  File "/usr/lib/python3.6/site-packages/django_redis/pool.py", line 64, in connect
    connection = self.get_connection(params)
  File "/usr/lib/python3.6/site-packages/django_redis/pool.py", line 75, in get_connection
    pool = self.get_or_create_connection_pool(params)
  File "/usr/lib/python3.6/site-packages/django_redis/pool.py", line 94, in get_or_create_connection_pool
    self._pools[key] = self.get_connection_pool(params)
  File "/usr/lib/python3.6/site-packages/django_redis/pool.py", line 107, in get_connection_pool
    pool = self.pool_cls.from_url(**cp_params)
  File "/usr/lib/python3.6/site-packages/redis/connection.py", line 916, in from_url
    return cls(**kwargs)
  File "/usr/lib/python3.6/site-packages/rediscluster/connection.py", line 146, in __init__
    self.nodes.initialize()
  File "/usr/lib/python3.6/site-packages/rediscluster/nodemanager.py", line 172, in initialize
    raise RedisClusterException("ERROR sending 'cluster slots' command to redis server: {0}".format(node))
rediscluster.exceptions.RedisClusterException: ERROR sending 'cluster slots' command to redis server: {'host': 'xxxxxxx.mc-redis-cache-v2.zzzzz.usw2.cache.amazonaws.com', 'port': '6379'}

I also tried passing "ssl_ca_certs": "/etc/ssl/certs/ca-certificates.crt" to CONNECTION_POOL_KWARGS and setting the location scheme to rediss still no luck

[devinnasar]

I'm having the same issue as @kashypAkash. In addition, I'm getting this error here:

app_1  | Traceback (most recent call last):
app_1  |   File "/env/lib/python3.8/site-packages/rediscluster/nodemanager.py", line 162, in initialize
app_1  |     cluster_slots = r.execute_command("cluster", "slots")
app_1  |   File "/env/lib/python3.8/site-packages/redis/client.py", line 755, in execute_command
app_1  |     return self.parse_response(connection, command_name, **options)
app_1  |   File "/env/lib/python3.8/site-packages/redis/client.py", line 768, in parse_response
app_1  |     response = connection.read_response()
app_1  |   File "/env/lib/python3.8/site-packages/redis/connection.py", line 638, in read_response
app_1  |     raise response
app_1  | redis.exceptions.ResponseError: NOAUTH Authentication required.

[devinnasar]

After some digging, I learned that unless an Elasticache Redis cluster is created with cluster mode/sharding, or if the cluser size is 1, Elasticache creates the cluster with a cluster_enabled: 0 setting. This causes redis-py-cluster to become confused when attempting to send the cluster slots command. In this case, as is @kashypAkash's issue, one simply configures Django-Redis as one would for a single Redis node:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": f"rediss://:password@master.your-elasticache-endpoint-cache.9iktx7.use1.cache.amazonaws.com:6379/0",
        "OPTIONS": {
            "CLIENT_CLASS":"django_redis.client.DefaultClient",
            "CONNECTION_POOL_KWARGS": {
                "ssl_cert_reqs": False
            }
        }
    },

Note that one must use the following conventions:

• use the rediss:// protocol to denote SSL connection
• make sure your password is prefixed with a colon in the url

[WisdomPill]

Is anyone of you interested in clearing out this doubt in the docs?

Maybe in a FAQ section

[konstankinollc]

FYI, gents you can add ssl_cert_reqs with none value to the connection params in REDIS_URL, in case there is not ability to change additional CACHE params in the code.

E.g.

rediss://:password@master.your-elasticache-endpoint-cache.9iktx7.use1.cache.amazonaws.com:6379/0?ssl_cert_reqs=none

Will be equivalent to the following configuration:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": f"rediss://:password@master.your-elasticache-endpoint-cache.9iktx7.use1.cache.amazonaws.com:6379/0",
        "OPTIONS": {
            "CONNECTION_POOL_KWARGS": {
                "ssl_cert_reqs": False
            }
        }
    },

See the source code here: https://docs.celeryproject.org/en/stable/_modules/celery/backends/redis.html