Read task tombstones from Celery database back-end, if configured

[chimaerase]

I'm using a database back-end for Celery so that I can have task tombstones for failure recovery purposes. I like Flower's default behavior of showing tasks that executed while it was running, but it's a bit misleading in my case since my Celery configuration establishes an expectation that a durable/all-inclusive store of task tombstones is available and visible in Flower. Experimenting with Flower's persistent setting shows that it appears to be saving its own history to file -- duplicating data stored in my database back-end, and still maybe missing some tasks if Flower was down at any point when tasks were processed.

I can see how the file-based store is an advantage for Flower users who don't use/want a Celery database back-end, but ideally the database back-end should be used when it's configured. A nice future add-on might be to manage the database store via Flower (e.g. to delete task tombstones of a certain age).

[matt-snider]

This is the behaviour I personally (and I imagine many others) would expect given that Flower is meant to tightly integrate with Celery.

[SamothRe]

Oh - that would be really great!

[wiltzius]

Agreed, I had just assumed that this is how Flower worked. I'm not sure how to best handle this in my server setup now. We have strict monitoring/auditing requirements for the jobs we run, so losing this state when the server restarts isn't OK... and periodically we rotate our entire server instances so file-based storage is a bit tricky too.

[viiicky]

Is this request accepted?

Asking this because I just came across a counter-intuitive comment by @ask at #143 which reads as:

Results should not be used to monitor a cluster, that would be devastating for performance, and events must not be used programatically because they're ephemeral and may happen out of order.

Just wondering if this request is being accepted upon to be implemented? Also, correct me if I have misuderstood the quoted lines.

[teyou]

sorry I may get lost. What's the latest update on this?

I have configured my flower to connect to broker & backend on the same redis server. However, whenever I restart the flower dashboard (running on AWS ECS server, sometimes there will be scale out and scale in process which will terminate old instance and create a new flower instance). All the task was lost even though the record still visible in redis