Redis: redis doc recommends using the SCAN command instead of the KEYS command.

[kislenko-artem]

There is this note on page https://master--redis-doc.netlify.app/commands/keys/

Warning: consider `KEYS` as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases.

I think to use SCAN command better.

Command KEYS is used in *.lua scripts. In this place:

        -- If there are no more ack groups for this worker, then
        -- remove it from the heartbeats set.
        local ack_queues = redis.call("keys", dead_worker_acks .. "*")
        if not next(ack_queues) then
            redis.call("zrem", heartbeats, dead_worker)
        end

And there is also a problem with lua-scripts if you use Redis-Cluster and pass RedisCluster as broker, there will error, because RedisCluster doesn't support scripts.

[pataquets]

New to dramatiq, never used it before, just browsing issue queue to getting to know if it fits my use case. Disclaimer: I've not dug deep in the problem and my dramatiq knowledge is measured in merely minutes.

However, the issue title quickly piqued my interest. KEYS is not meant for production. Been there, done that (on a previous life 😄). As Drupal DevOps on a previous work, I had to fix a somewhat similar issue which caused very nasty service crashes under heavy loads. Although Redis was used there as a caching backend (not queueing), the problem with KEYS looks like the same to me (apologies if it does not) and, in fact, SCAN was the solution, with some Redis server-side code (in Lua). Also, the client-side code is Drupal (PHP), but the meaty part is the server-side Lua code. Anyway, in case it might be of any help, I'm posting here the Drupal issue I created, which has also working code, which I think it might be useful, at least as far as the problem description goes. The issue is quite old (Drupal 7), but may be useful, anyway: https://www.drupal.org/project/redis/issues/2851625

I hope you don't mind the shameless plug. 😄 Hope it helps, and apologies if it does not.