agent icon indicating copy to clipboard operation
agent copied to clipboard

[Feature request] Allow configuration of traces in operator-spawned agent

Open RixTmobilender opened this issue 4 years ago • 13 comments
trafficstars

When using the grafana agent Operator, there's currently no way to populate the top level "traces" key inside the autogenerated config-file.

This means I can't configure my instrumented applications to send traces to the grafana agent and instead I have to push them directly to tempo's distributor which is non-ideal depending on the network setup. An additional workaround I'm currently using is manually declaring a deployment and service of the grafana agent with traces-only configuration, this deployment and service live along the operator-spawned agent which deals only in metrics and logs.

I wonder if it would be possible to allow arbitrary configuration merged into the agent config-file or if this feature would demand a CRD specific for traces.

RixTmobilender avatar Oct 28 '21 23:10 RixTmobilender

Hey, thanks for the feature request! I agree this would be nice to have, and we're generally planning for traces to be added into the operator eventually. We've been trickling features through so far: metrics came first, then logs, and currently we're working on integrations. Traces would be next up, and we'll use this issue for tracking when we finally implement it.

rfratto avatar Oct 29 '21 16:10 rfratto

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jan 09 '22 03:01 stale[bot]

I'm going to mark this as keepalive. We're definitely going to add support for traces in the operator, it's just a question of when.

rfratto avatar Jan 10 '22 12:01 rfratto

@rfratto Hello. I would be also interested in that feature - is there any ETA for the feature?

bartoszbehring-form3 avatar Mar 09 '22 12:03 bartoszbehring-form3

@bartoszbehring-form3 Hi! It's being looked at currently, but I don't have an ETA for when it might be available (it's probably at least 6-8 weeks away still)

rfratto avatar Mar 10 '22 16:03 rfratto

Been a while since this issue had some activity. But I am hoping it had some work done behind the scenes. Any update on this subject?

Siebjee avatar Jan 11 '23 15:01 Siebjee

Hi, it would be helpful for our org if Grafana Agent Operator supported trace collection and export :)

szymondurak-wr avatar Feb 14 '23 13:02 szymondurak-wr

I think this feature request should have higher priority, it is one of the few things keeping Grafana agent operator from being a complete observability collection stack for kubernetes right now I have to install open telemetry collector beside the operator just to collect traces

husseinraoouf avatar Mar 07 '23 18:03 husseinraoouf

Any updates on this?

FredrikAugust avatar Jun 07 '23 10:06 FredrikAugust

At the moment, the team has been focusing on Grafana Agent Flow as a way to unify the Grafana Agent ecosystem, including adding native operator support to it via the prometheus.operator.podmonitors and prometheus.operator.servicemonitors components.

We currently believe Flow is a better long-term solution, as it doesn't require duplicating our effort across two binaries, where one (ie the operator) needs a bespoke solution to configure and deploy things.

Never say never, but because we currently believe Flow is a better long-term solution for both maintainers an users, at this point it's unlikely that the operator will support traces.

Over the coming months we'll introduce tooling and documentation to help people who are interested migrate off of the operator onto Flow so they can take advantage of Flow's existing tracing support and support for consuming Prometheus Operator CRDs.

rfratto avatar Jun 08 '23 16:06 rfratto

If you are in a position of wanting to use the agent for traces in kubernetes presently, it is perfectly possible to make an agent deployment with our helm chart for that purpose.

captncraig avatar Jun 08 '23 17:06 captncraig

...2 years later...

another cons is related to faro, since you can't configure grafana agent operator to scrape traces and write to grafana tempo you can't also use grafana faro... because the agent will not delivery the content. (because you can't configure the operator)

sad

...2 years later...

karmops avatar Sep 03 '23 20:09 karmops

So, what's the current recommended way of configuring trace collection on a Kubernetes cluster? Operator plus a separate grafana agent just for traces? Flow? No way at all?

bsod90 avatar Oct 10 '23 00:10 bsod90