dramatiq icon indicating copy to clipboard operation
dramatiq copied to clipboard

Namespacing not used for RedisBackend results storage

Open coandco opened this issue 2 years ago • 0 comments

Issues

GitHub issues are for bugs. If you have questions, please ask them on the discussion board.

Checklist

  • [x] Does your title concisely summarize the problem?
  • [x] Did you include a minimal, reproducible example?
  • [x] What OS are you using?
  • [x] What version of Dramatiq are you using?
  • [x] What did you do?
  • [x] What did you expect would happen?
  • [x] What happened?

What OS are you using?

Ubuntu 20.04

What version of Dramatiq are you using?

1.13.0

What did you do?

I'm investigating dramatiq for my project, and I set it up like so:

redis_nodes = {'localhost': 6379}
redis_client = Redis(host='localhost', port=6379)
broker = RedisBroker(client=redis_client, namespace="{dramatiq}")
broker.add_middleware(Results(backend=RedisBackend(client=redis_client, namespace="{dramatiq}:results")))
dramatiq.set_broker(broker)

What did you expect would happen?

I expected that message results would get stored in a namespaced key in Redis -- that is to say, that the message results would be something like {dramatiq}:results:<message-id>.

What happened?

The results got stored without the namespace. Looking at the code, it creates a nice namespaced string in the form of %(namespace)s:%(queue_name)s:%(actor_name)s:%(message_id)s at https://github.com/Bogdanp/dramatiq/blob/master/dramatiq/results/backend.py#L146, then tosses that all away and takes the md5 hash at https://github.com/Bogdanp/dramatiq/blob/master/dramatiq/results/backend.py#L152. This results in non-namespaced md5 hashes getting written to the Redis server, which could lead to md5 collisions with other non-namespaced services using the same server. Also, this prevents results from being used with a Redis hash tag in the event the upstream server is a Redis Cluster.

coandco avatar Jul 11 '22 21:07 coandco