agent icon indicating copy to clipboard operation
agent copied to clipboard

Send observability signals to Kafka

Open arnitolog opened this issue 1 year ago • 9 comments

Request

Hello, Grafana Tempo can consume traces directly from Kafka already. Grafana Mimir merged the changes to do that recently (https://github.com/grafana/mimir/pull/6929). I think it will be good to have the ability to send all kinds of observability data from the agent to Kafka. Is it somewhere on the roadmap?

Use case

Kafka can be used as an intermediate buffer for all observability signals. So metrics scraping/ tracing, logs and profile collection, and actual data ingestion can go at their own pace. It also will prevent data loosing in case of overloaded ingesters. So data will not be discarded but delayed

arnitolog avatar Dec 15 '23 18:12 arnitolog