Set up grafana and log things

[yk]

• postgres metrics, table sizes, etc.
• backend metrics
• load balancer metrics
• frontend metrics

[andrewm4894]

Hey for frontend telemetry I can help set up https://posthog.com/ really really easy.

Basically just snippet and then get a lot of really good stuff for free basically.

[andrewm4894]

Related issue, just cross posting.

https://github.com/LAION-AI/Open-Assistant/issues/458

[abinmn]

I'd like to take up the Grafana setup and Postgres integration part

[AbdBarho]

@abinmn it is all yours

[andrewm4894]

I'd be interested in helping too, I work at a monitoring company (https://github.com/netdata/netdata)

Pretty sure I could get us a free sponsored space for monitoring via netdata, would be fairly easy to set up. Here is a demo space with a postgres node in it. https://app.netdata.cloud/spaces/netdata-demo/rooms/postgresql/

This could of course be in addition to grafana and Prometheus or in place or a mix. (We have good ml based anomaly detection - that's what I work on, and it's not just a wrapper on FB prophet :) like most others in the space).

Anyway would be happy to help too whatever tools we decide to use.

Any strong opinions on if the monitoring infra needs to be self hosted or open to using cloud stuff like grafana cloud, datadog, new relic etc. ?

I'd imagine a lot of them have decent open source programs that we could make use of to minimize overhead of maintaining monitoring infra ourselves (even though I'm game for that too tbh).

[andrewm4894]

made a PR here with example of adding local netdata dashboards for containers and redis, postgres, etc.

will try have a stab at something similar for Grafana and Prometheus in a different feature branch.