arq

arq copied to clipboard

In multi-tenant scenarios, Redis recommends using namespaces to segment tenants. This is especially useful for non-production environments with hosted Redis providers. Instead of paying for a new Redis instance for each environment, the key space can be segmented and assigned to an environment.

This issue is to request the ability to define an arbitrary key prefix or namespace that would allow 2 arq services to operate on a single Redis instance/cluster.

Some prior art:

• Django: KEY_PREFIX setting
• Celery: global_keyprefix setting

Could you explain why the queue_name option is not sufficient?

The prefix would need to be applied to every key in Redis. It's not clear to me that queue_name will be used for all these values?

https://github.com/samuelcolvin/arq/blob/ab2dda2011ab27007650c4918d3704f3bf7ac13d/arq/constants.py#L1-L6

+1 I faced the same issue too when having two separate worker codebases use the same redis db. The queue name prefix applied to only a small subset of keys generated. So it was hard for me even while using a redis gui browser (like redis insights) to figure out which entries belonged to which. A clean name spacing separation at the root level would be more ideal.