backend Investigate quicker ways to deploy monitoring scripts

Setting up daemons that write to node_exporter is time consuming

We can test netdata plugins after ooni/backend#268

Dec 10 '19 16:12 FedericoCeratto

Why not make the daemons that expose Prometheus metrics directly, rather than write out to the node_exporter?

Dec 10 '19 19:12 SuperQ

@SuperQ it would make the daemons significantly more complex, having to run a dedicated thread each to expose the metrics, and more burdensome to deploy, having to assign a unique port to each, allow incoming traffic and configure Prometheus to scrape each of them. It's difficult to justify when the code doing the real monitoring work is 1 line of bash. (e.g. finding and counting files, running one SQL query, ...)

Dec 10 '19 19:12 FedericoCeratto

The kinds of monitoring tasks you're talking about are already implemented by good quality code. For example, there are several SQL query exporters for exposing SQL query data.

Add monitoring endpoints to existing daemons is not "significantly more complex". These techniques are well documented and common.

Dec 10 '19 20:12 SuperQ