djangoproject.com icon indicating copy to clipboard operation
djangoproject.com copied to clipboard

Remove dependency on memcached

Open apollo13 opened this issue 5 years ago • 8 comments

We currently have a memcached and a redis cache configured. We should get rid of one software :)

apollo13 avatar Sep 12 '20 09:09 apollo13

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Oct 05 '22 10:10 stale[bot]

We should implement this.

carltongibson avatar Oct 05 '22 10:10 carltongibson

I don't know the real infrastructure in which the site is deployed. Which cache we are using in production: Memcached or Redis? Knowing this information we could open a correct PR.

pauloxnet avatar Oct 05 '22 11:10 pauloxnet

@pauloxnet As I read the report, the goal would be to use Redis.

CACHES = {
    'default': {
        'BACKEND': 'django_pylibmc.memcached.PyLibMCCache',
        'LOCATION': SECRETS.get('memcached_host', '127.0.0.1:11211'),
        'BINARY': True,
        'OPTIONS': {
            'tcp_nodelay': True,
            'ketama': True
        }
    },
    '`': {
        'BACKEND': 'redis_cache.RedisCache',
        'LOCATION': SECRETS.get('redis_host', 'localhost:6379'),
        'OPTIONS': {
            'DB': 2,
        },
    },
}

So maybe have the default cache use the built-in Redis backend, and remove the separate docs-pages — so we only have one backend in use (or one service at least).

We should be able to remove a few dependencies at that point.

@apollo13 — any guidance welcome 😉

carltongibson avatar Oct 05 '22 12:10 carltongibson

So yes https://github.com/django/djangoproject.com/blob/main/djangoproject/settings/prod.py#L15-L32 lists two caches. I'd argue that one of those can be removed. I think we might wanna remove memcached assuming we are using some special features from redis? So first and foremost it is a question of how we use those backends (code-wise) and then we can see about removing one.

apollo13 avatar Oct 05 '22 15:10 apollo13

From a code perspective I'd agree with @carltongibson - drop to a single cache so the docs & site use the same redis db. There isn't that much in the project using cache beyond the basics so a single instance would do it.

Unless of course, the metrics on the infrastructure show's theres a need for a split setup.

However, the built-in redis backend is new in django 4.0 right? And the site is still on 3.2.

On a side note, if the redis connection isn't using SSL it may also be worth looking at that - though that's environment variable based so I can't see if that's already happening. (The SSL connection uses rediss:// on port 6380.)

marksweb avatar Oct 16 '22 22:10 marksweb