Query timeout

Open aleksandarilic96 opened this issue 5 years ago • 1 comments

Hello everyone,

I'm having an issue with a query that is almost constantly timing out.

The query itself last max 15 sec and I have set up my config like this:

sql_exporter

global: scrape_timeout_offset: 30s

prometheus:

scrape_interval: 60s
scrape_timeout: 55s

I have gone through the documentation and issues but I'm sure I'm missing something.

Please let me know if I should change something or config it differently.

Thanks and Best Regards

Dec 02 '20 15:12 aleksandarilic96

It is kind of hard to tell what's going on without any logs. All I can tell from looking at your configuration is that the query has at most 25 seconds to complete (scrape_timeout - scrape_timeout_offset = 55s - 30s = 25s).

It could be that there are multiple Prometheus instances running the same query and the queries all get serialized. If so, I would set min_interval to 15s - 50s, so you serve cached metrics on some of those scrapes. May be worth doing regardless, a query that runs for 15 seconds is not something you want to run every couple of seconds. And regardless, I would drop the scrape_timeout_offset quite a bit. It's only there to cover network latency.

Did you look at your DB logs and saw that the query actually completes in 15 seconds when run by SQL Exporter?

Dec 02 '20 17:12 free