Expose prometheus metrics

[mrueg]

Is your feature request related to a problem? Please describe.

Currently it's difficult to track how localAI is used and if it's working. It would be good to have metrics on response times as well as errors. https://sre.google/sre-book/monitoring-distributed-systems/

Describe the solution you'd like Have a /metrics endpoint that exposes https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md

Describe alternatives you've considered Exporting metrics via json_exporter

Additional context

[mudler]

Hey @mrueg :wave:

nice to see you again! That sounds like a very useful addition, definitely something we want to add.

[sebastien-prudhomme]

OpenTelemetry is another option. You could have both metrics (compatible with OpenMetrics with a Prometheus exporter) and traces.

[soleblaze]

OTel is becoming the new defacto way to do it. I’ve been wanting a project where I can set this up. I’ll take a look at this sometime in the next week. Need to setup an environment to ingest them first.

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful :smile_cat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

Thank you for your feature request! We understand the importance of having metrics to monitor the performance and health of a system. We will consider adding a /metrics endpoint that exposes OpenMetrics metrics in our future releases. In the meantime, we recommend using OpenObservability's OpenMetrics implementation as a workaround. If you have any further questions or concerns, please don't hesitate to ask!

Sources:

• https://localai.io/index.html
• https://github.com/go-skynet/LocalAI/tree/master/examples/README.md
• https://github.com/go-skynet/LocalAI/tree/master/api/config/prediction.go
• https://localai.io/basics/news/index.html

[jespino]

I have an initial setup for adding metrics using OpenTelemetry. I'm limiting it to the bare minimum to introduce only what is needed for expose prometheus metrics, in the future we can expand that to add traces. What kind of metrics would you like to add there? The default Go metrics are there, but what would you consider interesting to include?