django-redis
django-redis copied to clipboard
Published
20 hours ago •
jazzband
jazzband
Unable to use TLS encryption
Hey folks,
actually I wanted to implement TLS encryption to connect my django application with redis but im failing with the following error (using without TLS work like a charm):
app | raise ConnectionError("Error while reading from socket: %s" %
app | redis.exceptions.ConnectionError: Error while reading from socket: (104, 'Connection reset by peer')
my settingy.py:
CACHES = {
'default': {
'BACKEND': 'django_prometheus.cache.backends.redis.RedisCache',
'LOCATION': [
'rediss://' + ':' + env.str('REDIS_PWD') + '@' + env.str('REDIS_HOST') + ':' + env.str('REDIS_PORT') + '/' + env.str('REDIS_CACHE_DB')
],
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
'SOCKET_CONNECT_TIMEOUT': 30, # seconds
'SOCKET_TIMEOUT': 30, # seconds
'COMPRESSOR': 'django_redis.compressors.zlib.ZlibCompressor',
'CONNECTION_POOL_KWARGS': {'max_connections': env.int('REDIS_MAX_CONN'),
'retry_on_timeout': True,
'ssl_ca_certs': '/etc/ssl/CAcert.pem'
},
'REDIS_CLIENT_KWARGS': {'skip_full_coverage_check': True,
'ssl': True,
'ssl_cert_reqs': None
}
}
}
}
This is how I start redis using docker:
redis:
image: redis:alpine
command: >
--requirepass ${REDIS_PWD}
--protected-mode no
--logfile "/var/log/redis-server.log"
--loglevel "verbose"
--tls-ca-cert-file /etc/ssl/CAcert.pem
--tls-key-file /etc/ssl/redis.key
--tls-cert-file /etc/ssl/redis.crt
--tls-dh-params-file /etc/ssl/dhparams.pem
--port 0
--tls-port 6379
--tls-auth-clients yes
--tls-protocols "TLSv1.2 TLSv1.3"
container_name: ${REDIS_HOST}
hostname: ${REDIS_HOST}
networks:
- backend
ports:
- ${REDIS_PORT}
labels:
- traefik.enable=false
volumes:
- ./hard_secrets/ssl/redis:/etc/ssl
- ./runtimedata/log/redis:/var/log
The redis log is showing the following:
1:C 22 Jan 2021 23:15:23.486 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 22 Jan 2021 23:15:23.486 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 22 Jan 2021 23:15:23.486 # Configuration loaded
1:M 22 Jan 2021 23:15:23.488 * Running mode=standalone, port=6379.
1:M 22 Jan 2021 23:15:23.488 # Server initialized
1:M 22 Jan 2021 23:15:23.488 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 22 Jan 2021 23:15:23.488 * Ready to accept connections
1:M 22 Jan 2021 23:15:38.848 - Accepted 172.21.0.7:36354
1:M 22 Jan 2021 23:15:38.848 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (conn: fd=8)
1:M 22 Jan 2021 23:15:39.402 - Accepted 172.21.0.7:36390
1:M 22 Jan 2021 23:15:39.402 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (conn: fd=8)
1:M 22 Jan 2021 23:15:39.945 - Accepted 172.21.0.9:36502
I create my Certificate using the following script
https://pastebin.com/mH8YpERZ
Does smb can provide any help onto this?
Kind regards :D
I want to add to this. Wanted to get our local development in line with what we deploy on heroku.
Trying to setup redis6 with ssl support on docker using : https://github.com/allen-munsch/docker-redis-ssl-example
BASE_REDIS_URL = f"rediss://{os.environ.get('REDIS_CFG_HOST')}:6379"
REDIS_URL = BASE_REDIS_URL + "?ssl_cert_reqs=none"
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": REDIS_URL,
"OPTIONS": {"CLIENT_CLASS": "django_redis.client.DefaultClient"},
}
}
web | --- Logging error ---
web | Traceback (most recent call last):
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 32, in _decorator
web | return method(self, *args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 67, in set
web | return self.client.set(*args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/client/default.py", line 145, in set
web | raise ConnectionInterrupted(connection=client, parent=e)
web | django_redis.exceptions.ConnectionInterrupted: Redis ConnectionError: Error while reading from socket: (1, '[SSL] tlsv13 alert certificate required (_ssl.c:2309)')
web |
web | During handling of the above exception, another exception occurred:
web |
web | Traceback (most recent call last):
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 994, in emit
web | msg = self.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 840, in format
web | return fmt.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 580, in format
web | s = self.formatMessage(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 549, in formatMessage
web | return self._style.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 391, in format
web | return self._fmt % record.__dict__
web | File "/venv/lib/python3.6/site-packages/django/utils/functional.py", line 246, in inner
web | self._setup()
web | File "/venv/lib/python3.6/site-packages/django/utils/functional.py", line 382, in _setup
web | self._wrapped = self._setupfunc()
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/middleware.py", line 23, in <lambda>
web | request.user = SimpleLazyObject(lambda: get_user(request))
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/middleware.py", line 11, in get_user
web | request._cached_user = auth.get_user(request)
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 177, in get_user
web | user_id = _get_user_session_key(request)
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 60, in _get_user_session_key
web | return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 65, in __getitem__
web | return self._session[key]
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 238, in _get_session
web | self._session_cache = self.load()
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/cached_db.py", line 38, in load
web | self._cache.set(self.cache_key, data, self.get_expiry_age(expiry=s.expire_date))
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 39, in _decorator
web | raise e.parent
web | File "/venv/lib/python3.6/site-packages/django_redis/client/default.py", line 139, in set
web | return bool(client.set(nkey, nvalue, nx=nx, px=timeout, xx=xx))
web | File "/venv/lib/python3.6/site-packages/redis/client.py", line 1801, in set
web | return self.execute_command('SET', *pieces)
web | File "/venv/lib/python3.6/site-packages/redis/client.py", line 898, in execute_command
web | conn = self.connection or pool.get_connection(command_name, **options)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 1203, in get_connection
web | if connection.can_read():
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 734, in can_read
web | return self._parser.can_read(timeout)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 321, in can_read
web | return self._buffer and self._buffer.can_read(timeout)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 231, in can_read
web | raise_on_timeout=False)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 223, in _read_from_socket
web | (ex.args,))
web | redis.exceptions.ConnectionError: Error while reading from socket: (1, '[SSL] tlsv13 alert certificate required (_ssl.c:2309)')
web | Call stack:
web | File "/venv/lib/python3.6/site-packages/eventlet/greenthread.py", line 221, in main
web | result = function(*args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/gunicorn/workers/geventlet.py", line 115, in handle
web | super().handle(listener, client, addr)
web | File "/venv/lib/python3.6/site-packages/gunicorn/workers/base_async.py", line 55, in handle
web | self.handle_request(listener_name, req, client, addr)
web | File "/venv/lib/python3.6/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request
web | respiter = self.wsgi(environ, resp.start_response)
web | File "/venv/lib/python3.6/site-packages/dj_static.py", line 83, in __call__
web | return self.application(environ, start_response)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
web | response = self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 130, in get_response
web | response = self._middleware_chain(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
web | response = get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
web | response = response or self.get_response(request)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 49, in inner
web | response = response_for_exception(request, exc)
web | File "/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 119, in response_for_exception
web | exc_info=sys.exc_info(),
web | File "/venv/lib/python3.6/site-packages/django/utils/log.py", line 230, in log_response
web | exc_info=exc_info,
web | Message: '%s: %s'
web | Arguments: ('Internal Server Error', '/')
redis | 8:M 19 May 2021 19:37:38.465 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
redis | 8:M 19 May 2021 19:37:38.601 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
redis | 8:M 19 May 2021 19:37:42.264 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
web | --- Logging error ---
redis | 8:M 19 May 2021 19:37:42.384 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
web | Traceback (most recent call last):
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 32, in _decorator
web | return method(self, *args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 67, in set
web | return self.client.set(*args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/client/default.py", line 145, in set
web | raise ConnectionInterrupted(connection=client, parent=e)
web | django_redis.exceptions.ConnectionInterrupted: Redis ConnectionError: Error while reading from socket: ('timed out',)
web |
web | During handling of the above exception, another exception occurred:
web |
web | Traceback (most recent call last):
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 994, in emit
web | msg = self.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 840, in format
web | return fmt.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 580, in format
web | s = self.formatMessage(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 549, in formatMessage
web | return self._style.format(record)
web | File "/usr/local/lib/python3.6/logging/__init__.py", line 391, in format
web | return self._fmt % record.__dict__
web | File "/venv/lib/python3.6/site-packages/django/utils/functional.py", line 246, in inner
web | self._setup()
web | File "/venv/lib/python3.6/site-packages/django/utils/functional.py", line 382, in _setup
web | self._wrapped = self._setupfunc()
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/middleware.py", line 23, in <lambda>
web | request.user = SimpleLazyObject(lambda: get_user(request))
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/middleware.py", line 11, in get_user
web | request._cached_user = auth.get_user(request)
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 177, in get_user
web | user_id = _get_user_session_key(request)
web | File "/venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 60, in _get_user_session_key
web | return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 65, in __getitem__
web | return self._session[key]
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 238, in _get_session
web | self._session_cache = self.load()
web | File "/venv/lib/python3.6/site-packages/django/contrib/sessions/backends/cached_db.py", line 38, in load
web | self._cache.set(self.cache_key, data, self.get_expiry_age(expiry=s.expire_date))
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 39, in _decorator
web | raise e.parent
web | File "/venv/lib/python3.6/site-packages/django_redis/client/default.py", line 139, in set
web | return bool(client.set(nkey, nvalue, nx=nx, px=timeout, xx=xx))
web | File "/venv/lib/python3.6/site-packages/redis/client.py", line 1801, in set
web | return self.execute_command('SET', *pieces)
web | File "/venv/lib/python3.6/site-packages/redis/client.py", line 898, in execute_command
web | conn = self.connection or pool.get_connection(command_name, **options)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 1203, in get_connection
web | if connection.can_read():
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 734, in can_read
web | return self._parser.can_read(timeout)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 321, in can_read
web | return self._buffer and self._buffer.can_read(timeout)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 231, in can_read
web | raise_on_timeout=False)
web | File "/venv/lib/python3.6/site-packages/redis/connection.py", line 223, in _read_from_socket
web | (ex.args,))
web | redis.exceptions.ConnectionError: Error while reading from socket: ('timed out',)
redis | 8:M 19 May 2021 19:37:42.505 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
web | --- Logging error ---
redis | 8:M 19 May 2021 19:37:42.624 # Error accepting a client connection: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
web | Traceback (most recent call last):
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 32, in _decorator
web | return method(self, *args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/cache.py", line 67, in set
web | return self.client.set(*args, **kwargs)
web | File "/venv/lib/python3.6/site-packages/django_redis/client/default.py", line 145, in set
web | raise ConnectionInterrupted(connection=client, parent=e)
web | django_redis.exceptions.ConnectionInterrupted: Redis ConnectionError: Error while reading from socket: ('timed out',)
web |
web | During handling of the above exception, another exception occurred:
Hello @venomone, sorry for the late response.
Have you solved your issue?
Is #353 what you were looking for?
Hey @allen-munsch and @WisdomPill I'm having an issue with a premium-0 Heroku Redis instance after I upgraded from Redis 5 to Redis 6.2; the latter requires SSL.
Can you give an example of how the Django setup should look to get this working? This is what I have currently and my cache_page() API call is now returning 500. This is based on the recommendation from Heroku docs.
CACHES = {
'default': {
'BACKEND': 'django_redis.cache.RedisCache',
# Use secure Redis URL available in Redis 6+.
# For premium instance on Heroku, this is a rediss:// url.
'LOCATION': os.getenv('REDIS_URL'),
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
'CONNECTION_POOL_KWARGS': {
'ssl_cert_reqs': None
},
},
}
}
It seems to work now actually; I added the REDIS_CLIENT_KWARGS settings.
CACHES = {
'default': {
'BACKEND': 'django_redis.cache.RedisCache',
# Use secure Redis URL available in Redis 6+.
# For premium instance on Heroku, this is a rediss:// url.
'LOCATION': os.getenv('REDIS_URL'),
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
'CONNECTION_POOL_KWARGS': {
'ssl_cert_reqs': None
},
'REDIS_CLIENT_KWARGS': {
'ssl': True,
'ssl_cert_reqs': None
},
},
}
}
@getup8 Hi! Did you work out why you needed to add REDIS_CLIENT_KWARGS too, and not just CONNECTION_POOL_KWARGS. From code inspection, it seems that setting the latter on its own should be sufficient, since the options are passed down to the client. Or was the connection pool disabled in your app?
@getup8 what you have initially looks like it should work, do you mind testing to see if you actually need the REDIS_CLIENT_KWARGS? I walked through the code and it really doesn't look like you should. You can also see what connection gets created for both if you use the following:
>>> from django.core.cache import caches
>>> # 'default' should match the Redis CACHES
>>> db = caches['default'].client.get_client()
# or redis.connection.Connection if using redis://
>>> db.connection_pool.connection_class
redis.connection.SSLConnection
>>> db.connection_pool.connection_kwargs
{
...,
'ssl_cert_reqs': None,
...,
}
For anyone who got here attempting TSL parity in a local docker env vs Heroku
Client certificate authentication
By default, Redis uses mutual TLS and requires clients to authenticate with a valid certificate (authenticated against trusted root CAs specified by ca-cert-file or ca-cert-dir).
You may use tls-auth-clients no to disable client authentication.
https://redis.io/docs/manual/security/encryption/
So your redis command becomes something like redis-server --tls-port 6379 --port 0 --tls-cert-file /tls/redis.crt --tls-key-file /tls/redis.key --tls-auth-clients no
