celery-prometheus-exporter icon indicating copy to clipboard operation
celery-prometheus-exporter copied to clipboard

How to extend for multiple queue monitoring?

Open miguelHx opened this issue 7 years ago • 5 comments

Hello, I am curious as to how we can extend to expose metrics for multiple queues. Could you point me to where I can start? I am kind of a newbie at this so any guidance would be appreciated.

miguelHx avatar Jun 25 '18 22:06 miguelHx

Sorry, didn't have the time to look into your request :( Did you already solve it on your own?

zerok avatar Jul 07 '18 14:07 zerok

I saw from issue #19 you mention that the queue name was there in the task class. I thought that maybe I should start there and work to get multiple queue monitoring. So that's why I closed the issue. Going to re-open, though. How can we extend for multiple queues?

miguelHx avatar Jul 08 '18 01:07 miguelHx

I‘m currently afk, but I’d guess that the information about the queue is also included in the submitted event. You could then add the queue as additional label in the _incr*

I hope this helps 🙂

zerok avatar Jul 09 '18 16:07 zerok

I'll look into that, thanks!

miguelHx avatar Jul 09 '18 20:07 miguelHx

I think this can be done by capturing and saving the task-sent event:

task-sent(uuid, name, args, kwargs, retries, eta,expires, queue, exchange, routing_key, root_id,parent_id)

... which can then be used to lookup queue, exchange and routing_key if desired, by uuid, then later discarded.

Anyone looked at that yet? Bit of work there to adapt current code, thinking not done yet ...

javabrett avatar Aug 15 '18 05:08 javabrett