Start exporter even if nginx is not running

[matejzero]

Is your feature request related to a problem? Please describe.

At the moment, if nginx is not running, exporter won't start with error Could not create Nginx Client: failed to get http://127.0.0.1:8080/stub_status: Get http://127.0.0.1:8080/stub_status: dial tcp 127.0.0.1:8080: connect: connection refused.

This is not typical for other Prometheus exporters, as they don't have this limit. There are cases, when server can boot up but due to an error in configuration, nginx won't start, user forgot to set systemd to start nginx, there was a slow start,... and exporter won't as well. User then, when he gets paged, need to fix the error, start nginx and also remember to start exporter as well.

Describe the solution you'd like

Looking at apache_exporter, redis_exporter, mysql_exporter and many other, they all expose <exporter_name>_up metric, like apache_up, mysql_up. In that case, one can use up metric to monitor if exporter is running or not and nginx_up to alert if nginx is not running.

In that case, I don't have to worry about successfully starting nginx before I start nginx exporter.

[matejzero]

If one wants to aboud having 2 metrics (up and nginx_up), you could fail with 500 upon unsuccessful scrape, like described here: https://www.robustperception.io/failing-a-scrape-with-the-prometheus-go-client

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.

[eheydrick]

Any chance this could be revisited? It would be ideal if the exporter could start up even if nginx isn't running.

[matejzero]

This is the right way of doing it as exporters also (quite ofter) show metric like <service>_up which can tell us if nginx is running. Using only up metric is not good enough as one doesn't know if exporter is not running or nginx is not running.

[pleshakov]

Could this feature https://github.com/nginxinc/nginx-prometheus-exporter/pull/37 be utilized as a workaround the the time being? although please note that during retrying, the Prometheus endpoint :9113 is not created yet.

[pleshakov]

Does the following requirements sufficiently address the problem?

(1) If the exporter cannot connect to NGINX during start

(a) it doesn’t exit

(b) it shows the nginx_up metric set to 0:

# HELP nginx_up Status of the last metric scrape
# TYPE nginx_up gauge
nginx_up 0
# HELP nginxexporter_build_info Exporter build information
# TYPE nginxexporter_build_info gauge
nginxexporter_build_info{commit="3f927bafecf3e5eeff8c83bf2f5e9bf0d4a89624",date="",version="0.8.0"} 1

(2) When NGINX becomes available, the exporter shows metrics as usual:

# HELP nginx_connections_accepted Accepted client connections
# TYPE nginx_connections_accepted counter
nginx_connections_accepted 5.3421854e+07
# HELP nginx_connections_active Active client connections
# TYPE nginx_connections_active gauge
nginx_connections_active 165
# HELP nginx_connections_handled Handled client connections
# TYPE nginx_connections_handled counter
nginx_connections_handled 5.3421854e+07
# HELP nginx_connections_reading Connections where NGINX is reading the request header
# TYPE nginx_connections_reading gauge
nginx_connections_reading 0
# HELP nginx_connections_waiting Idle client connections
# TYPE nginx_connections_waiting gauge
nginx_connections_waiting 162
# HELP nginx_connections_writing Connections where NGINX is writing the response back to the client
# TYPE nginx_connections_writing gauge
nginx_connections_writing 2
# HELP nginx_http_requests_total Total http requests
# TYPE nginx_http_requests_total counter
nginx_http_requests_total 7.4669107e+07
# HELP nginx_up Status of the last metric scrape
# TYPE nginx_up gauge
nginx_up 1
# HELP nginxexporter_build_info Exporter build information
# TYPE nginxexporter_build_info gauge
nginxexporter_build_info{commit="3f927bafecf3e5eeff8c83bf2f5e9bf0d4a89624",date="",version="0.8.0"} 1

[matejzero]

The last answer would be perfect. I think that is one of the best practice options for exporters and used in lots of exporters. Thumbs up from my side.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.

[pjmpsu]

Could this be revisited?