airflow-prometheus-exporter icon indicating copy to clipboard operation
airflow-prometheus-exporter copied to clipboard

Fix dag duration metric

Open jward-bw opened this issue 4 years ago • 9 comments

The filter was causing an implicit cartesian join on TaskInstance in the outer query, making this metric appear multiple times for a single dag. I have moved the filter into the sub-query which is already joined with TaskInstance to prevent this cartesian join.

@abhishekray07 I hope you can take a look at this soon.

jward-bw avatar Apr 01 '20 10:04 jward-bw

This PR addresses this issue: https://github.com/robinhood/airflow-prometheus-exporter/issues/24

jward-bw avatar Apr 01 '20 10:04 jward-bw

+1 for this fix, we have ~28K task instances, and "dag_run_duration" metric gets duplicated 28K times. @ayush-san @abhishekray07

mshalak-nix avatar May 07 '20 12:05 mshalak-nix

Any idea when these changes will be merged and released @ayush-san @abhishekray07?

drboyer avatar May 28 '20 19:05 drboyer

@drboyer I don't have merging rights. Waiting for @abhishekray07 to merge and release these changes

ayush-san avatar Jun 05 '20 11:06 ayush-san

We're also experiencing performance issues, after a week the metrics endpoint takes something like 20 seconds to respond. When do you think this fix will be merged?

dusty73 avatar Jul 09 '20 13:07 dusty73

The owner of this repo seems to be inactive, so you'd do better just forking and applying this patch yourself.

jward-bw avatar Jul 09 '20 13:07 jward-bw

FYI, issue #26 is also related to this PR.

azwjp avatar Oct 08 '20 01:10 azwjp

Hello just bumping this PR to see if any upstream maintainers will be able to merge this soon :)

ring-pete avatar Feb 08 '21 22:02 ring-pete

For anyone having performance or query response time issues with this exporter, this change will fix those.

jonathonbattista avatar Sep 10 '21 19:09 jonathonbattista