vault-secrets-operator icon indicating copy to clipboard operation
vault-secrets-operator copied to clipboard

VSO constantly rotates secret when VaultStaticSecret template text is empty string

Open JackCheng01 opened this issue 1 year ago • 3 comments

Describe the bug VSO constantly rotates secret when VaultStaticSecret template text is empty string , and triggers rollout restart for target resource.

To Reproduce Steps to reproduce the behavior:

  1. create VaultStaticSecret having empty text in templates e.g.
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  annotations:
  name: empty-text
  namespace: default
spec:
  destination:
    create: true
    name: empty-text
    overwrite: true
    transformation:
      excludeRaw: true
      excludes:
      - .*
      templates:
        emptyString:
          text: ""
        emptyStringFromGoTemplte:
          text: '{{- printf "%s" "" -}}'
    type: Opaque
  hmacSecretData: true
  mount: dummy
  namespace: dummy
  path: dummy
  refreshAfter: 30s
  rolloutRestartTargets:
  - kind: Deployment
    name: dummy
  type: kv-v2
  1. Monitor events for created VaultStaticSecret
k events --for vaultstaticsecret.secrets.hashicorp.com/empty-text
LAST SEEN   TYPE     REASON                    OBJECT                         MESSAGE
31s         Normal   SecretSynced              VaultStaticSecret/empty-text   Secret synced
6s          Normal   RolloutRestartTriggered   VaultStaticSecret/empty-text   Rollout restart triggered for {Deployment dummy}
6s          Normal   SecretRotated             VaultStaticSecret/empty-text   Secret synced

Expected behavior

VSO should not rotate secret for VaultStaticSecret when there's no solid change.

Environment

  • Kubernetes version:
    • Server Version: v1.29.1-gke.1589020
  • vault-secrets-operator version: 0.8.1.0

JackCheng01 avatar Sep 02 '24 06:09 JackCheng01

We are seeing a similar behaviour with the following VaultStaticSecret when the referenced secret in vault has no keys (empty value {}).

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  creationTimestamp: '2024-09-20T11:48:27Z'
  finalizers:
    - vaultstaticsecret.secrets.hashicorp.com/finalizer
  generation: 1
spec:
  destination:
    create: true
    name: api.secrets
    overwrite: false
    transformation:
      excludeRaw: true
  hmacSecretData: true
  mount: development
  path: my-secret
  rolloutRestartTargets:
    - kind: Deployment
      name: api
  type: kv-v2
  vaultAuthRef: api

Events:

LAST SEEN              TYPE      REASON                    OBJECT                          MESSAGE
37m (x966 over 14h)    Normal    RolloutRestartTriggered   VaultStaticSecret/api.secrets   Rollout restart triggered for {Deployment api}
29m (x6 over 33m)      Normal    RolloutRestartTriggered   VaultStaticSecret/api.secrets   Rollout restart triggered for {Deployment api}
29m (x6 over 33m)      Normal    SecretRotated             VaultStaticSecret/api.secrets   Secret synced

A rollout restart is being triggered every 1 min or so, you can see in the first event it was triggered 966 times during a 14h period.

Deleting the secret my-secret or setting a dummy key (e.g. FOO=bar) does prevent the restarts.

Kubernetes version: v1.28.12-eks-a18cd3a VSO version (helm chart): we have seen this issue in v0.7.1 and v0.8.1

alitto avatar Sep 26 '24 12:09 alitto

Any update on this? I bumped into exactly this after adding

transformation:
  excludeRaw: true

The presence of some empty secrets triggers an infinite restart of Pods.

ollaw avatar Oct 21 '25 08:10 ollaw

Hi @benashz - I saw you recently worked on a similar issue When you have a moment, could you take a look at this one? It seems related.

luislongom avatar Oct 21 '25 08:10 luislongom

Hey we're observing the same issue on VSO 0.9.1.

Update: updating to VSO 1.0.1 does not fix the same issue

sanadhis avatar Nov 19 '25 16:11 sanadhis