Add support for backend telemetry (tracing & metrics)

[zacps]

This uses the opentelemetry SDK to provide support for traces and two simple metrics (a conversations count and a messages count).

The telemetry entrypoint is built separately (unfortunately it does not seem like there is a method for doing this in one build as sveltekit overrides rollupOptions).

As written the metrics may be too high cardinality for some use cases (as they are keyed per user), but these attributes can always be dropped in a collector.

Fixes #692, #633

[antonioramos1]

Very happy to see this contribution! I was also missing having better observability since we currently have little visibility on how many users we have and how they are using chat-ui

[antonioramos1]

Hi @nsarrazin if you dont mind me asking, are we looking to merge this PR or to support observability in a different way?

[nsarrazin]

Hey so I was working on adding support for prometheus metrics because we need them for HuggingChat but I think we can probably do it with this PR! Thanks for the contribution. 🚀

I'd like to have a /metrics route on a different port which exports metrics in a format compatible with prometheus. I think that's doable with opentelemetry, will dive into this

[nsarrazin]

The type checking error seems to be reported here https://github.com/open-telemetry/opentelemetry-js/issues/3944

[zacps]

Hey so I was working on adding support for prometheus metrics because we need them for HuggingChat but I think we can probably do it with this PR!

It looks like this is possible via the prometheus exporter. If we add this I would like to keep the opentelemetry exporter as an option.

flowchart LR
    c["chat-ui"]-->|Prometheus|m["/metrics"]

Alternatively we could only support otlp exports and anyone who wanted a prometheus endpoint could get it through the opentelemetry-collector:

flowchart LR
    chat-ui --> |OTLP|opentelemetry-collector-->|Prometheus|/metrics
    opentelemetry-collector-->|prometheus-remote-write|endpoint
    

In my experience with opentelemetry so far using a collector has been pretty useful to add/transform labels, drop healthcheck data, etc