dragonfly-operator icon indicating copy to clipboard operation
dragonfly-operator copied to clipboard

ServiceMonitor doens't work out of the box

Open maxpain opened this issue 1 year ago • 7 comments

Hello. I've deployed the dragonfly-operator Helm chart with the following values:

serviceMonitor:
  enabled: true

But my VMAgent can't scrape metrics:

unexpected status code returned when scraping "http://10.244.2.242:8443/metrics": 400; expecting 200; response body: "Client sent an HTTP request to an HTTPS server.\n"

maxpain avatar Nov 03 '24 13:11 maxpain

Thanks for reporting!

I can try testing it out, but in the meanwhile we have something tested around metric collection of Dragonfly at https://www.dragonflydb.io/docs/managing-dragonfly/operator/prometheus-guide

Pothulapati avatar Nov 07 '24 17:11 Pothulapati

We are facing the same problem.

As far as I see, the ServiceMonitor is targeting the port 8443 where as in the pod configuration the parameter --metrics-bind-address=127.0.0.1:8080 is set with a other port.

joggeli34 avatar Nov 15 '24 13:11 joggeli34

Also see: https://github.com/dragonflydb/dragonfly-operator/issues/242

jonathon2nd avatar Nov 26 '24 17:11 jonathon2nd

I have this problem too

To fix it, you need to change the logic:

  1. start operator pod with -metrics-bind-address=0.0.0.0:8080
  2. this port(8080) must set in ports of deployment
  3. serviceMonitor is targeted to this port

And why don't you use podMonitor instead serviceMonitor? This operator deployment doesn't seem to require the service, as it doesn't accept traffic.

nikatar avatar Dec 17 '24 15:12 nikatar

Just ran into this issue, is there a workaround for now until a fix is merged?

caycehouse avatar Feb 08 '25 21:02 caycehouse

Run into the same issue. Is there any intention of fixing this? I believe it should be as simple as changing the binding address to 0.0.0.0:8080 in case servicemonitor is enabled

rubasace avatar Feb 23 '25 14:02 rubasace

Same questions: Why not use a PodMonitor - and how can Prometheus connect when the bind address is set to 127.0.0.1?

larsskj avatar May 20 '25 13:05 larsskj