dragonfly-operator icon indicating copy to clipboard operation
dragonfly-operator copied to clipboard

Service manifests constantly updated

Open XLordalX opened this issue 5 months ago • 7 comments

We are running into rate limits with our cloud provider due to dragonfly service resources constantly being updated by the dragonfly operator, even when there's no changes.

Is there any reason to update the service resource constantly even when there are no changes?

XLordalX avatar Jul 22 '25 11:07 XLordalX

I'm also facing this. Constant reconciling from the operator logs, and service is constantly updating and reconciling in my LB when there should be no changes.

CR manifest are as below:

apiVersion: dragonflydb.io/v1alpha1
kind: Dragonfly
metadata:
  name: my-dragonfly-1
  namespace: namespace
spec:
  env:
   - name: DEBUG
     value: "true"
  tlsSecretRef: 
    name: my-dragonfly-1-server-tls
  replicas: 4
  resources:
    requests:
      cpu: 1
      memory: 8Gi
  serviceSpec:
    type: LoadBalancer
    annotations:
      external-dns.alpha.kubernetes.io/hostname: "my-dragonfly-1.mydomain"
      external-dns.alpha.kubernetes.io/ttl: "3600"
      metallb.universe.tf/LoadBalancerIPs: "10.10.10.10"
  authentication:
    passwordFromSecret:
      name: dragonflydb-1
      key: password
  snapshot:
    persistentVolumeClaimSpec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 25Gi
      storageClassName: px
    cron: "0 0 * * *"
  affinity: 
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                  - dragonfly
          topologyKey: "kubernetes.io/hostname"

sindef avatar Jul 26 '25 13:07 sindef

Also hitting this problem.

cardboardpig avatar Jul 29 '25 03:07 cardboardpig

I also have a similar issue when using the tailscale operator. It seems like its a battle between operators. Tailscale operator wants to add a finalizer on the service and DragonflyDB operator removes it instantly creating an endless loop.

yo-l1982 avatar Aug 01 '25 08:08 yo-l1982

I did some digging into this. Any changes in the Service manifest will trigger a reconciliation of the Service resource and restore it to this operator desired state of the service, same goes for the statefulset. This could be a bit blocking when using in combination with other operators like in my case the tailscale operator.

I would argue(humbly with my limited knowledge) that the event added on the below line for the service is quite unnecessary.

https://github.com/dragonflydb/dragonfly-operator/blob/9874757bf4606ced0cd3bb679c2bb36b5d4f3edc/internal/controller/dragonfly_controller.go#L143

I can see the risk that something is manually or by other operators changed to break this operator but that would be quite unlikely right?

yo-l1982 avatar Aug 06 '25 14:08 yo-l1982

@yo-l1982 @XLordalX @sindef, here is a PR to fix it.

https://github.com/dragonflydb/dragonfly-operator/pull/339

akhilesh78 avatar Aug 08 '25 19:08 akhilesh78

Also hitting this problem.

https://github.com/dragonflydb/dragonfly-operator/pull/339

aagarwal-apexanalytix avatar Aug 08 '25 19:08 aagarwal-apexanalytix

Also hitting this problem.

#339

Hey, looks like you closed the PR?

sindef avatar Sep 04 '25 12:09 sindef