Store Celery task events in a datastore like Redis

[dmorina]

Is your feature request related to a problem? Please describe. Storing a large number of tasks e.g. 1M+ in memory makes Flower super slow. Additionally, given the fact that internally it uses an LRUCache whenever we schedule a ton of tasks (more than the in memory cache holds) we lose information about the tasks, old tasks that have been scheduled but not picked up by workers yet will be cleared out (this is common with lazy-queues that are consumed slowly).

Describe the solution you'd like I think it would be great to shift the cache to a datastore like Redis, this would make Flower more responsive and allow for storing task events as long as they are needed.

Would you be open to something like that?