grafana_monitoring_service example compatibility with v4.0

[that-rahul-guy]

Hello all, I have been following the prometheus-grafana integration example few months ago. I was on v1.59.dev3. Now I check that evidently has been updated a lot since then. So I wish to update and want to know if the mentioned example work for evidently v4.0 and above? If not then what changes will I have to implement for same (the code logic for calculating metrics) ? Please guide and let me know if that example needs to be updated so I can contribute to it simultaneously.

[elenasamuylova]

Hi @that-rahul-guy!

This is indeed an old example, with each is not currently maintained. We made several major changes to the library since - specifically, completely rewrote how metrics are computed, replacing analyzers for metrics. So, updating the example will require significant work.

If you are looking for an example of near real-time integration where you can send the predictions directly from the ML monitoring service, you can check out this new example with a collector service. Docs: https://docs.evidentlyai.com/user-guide/monitoring/collector_service Example code: https://github.com/evidentlyai/evidently/tree/main/examples/integrations/collector_service

In this scenario, you self-host the Evidently monitoring UI instead of Grafana - to visualize the metrics over time. Quickstart for ML monitoring is here: https://docs.evidentlyai.com/get-started/tutorial-monitoring

We’d recommend it as a suggested course of action since as a team, we are currently focused on this monitoring functionality - it gets all the attention and support :)

If you are keen on using Grafana, you can look for an up-to-date example here: https://github.com/evidentlyai/evidently/tree/main/examples/integrations/postgres_grafana_batch_monitoring

However, it follows a different architecture compared to the old one: you run batch monitoring jobs, compute metrics using Evidently, log to Postgres, and visualize on a Grafana dashboard.