o11y

[cirocosta]

so,

1. wire prometheus to the endpoints provided by the internal headless-service that's used for the sts
2. write grafana to promethes
3. write the monerod-sidecar

[cirocosta]

oh, how about multi-cluster?

.. thanos is usually the answer for federated prometheus, but I think the answer for now could be a simple "grafana somewhere pointing at the known secured prometheus endpoints".

[cirocosta]

so ... I was doing rpc result --> prometheus value by hand, but it seems like we could take a better approach of instead writing a custom Collector that once hit by Prometheus, we make the calls we need and then rather than doing field-by-field conversions for methods like get_info, instead look at the json tags for snake-case names, and then based on the types, create the corresponding prometheus.Metric objects

ps.: with collectors you're accumulating/collecting once prometheus hits you, which I think makes sense for what we're trying to accomplish