redpanda-operator
redpanda-operator copied to clipboard
Published
20 hours ago •
redpanda-data
redpanda-data
Redpanda Operator spams with sh.helm.release secrets when workload unhealthy
trafficstars
When deployment unhealthy, the operator creates many secret objects:
sh.helm.release.v1.redpanda.v100 helm.sh/release.v1 1 82m
sh.helm.release.v1.redpanda.v101 helm.sh/release.v1 1 67m
sh.helm.release.v1.redpanda.v102 helm.sh/release.v1 1 51m
sh.helm.release.v1.redpanda.v103 helm.sh/release.v1 1 36m
sh.helm.release.v1.redpanda.v104 helm.sh/release.v1 1 21m
sh.helm.release.v1.redpanda.v105 helm.sh/release.v1 1 6m3s
sh.helm.release.v1.redpanda.v2 helm.sh/release.v1 1 14d
sh.helm.release.v1.redpanda.v31 helm.sh/release.v1 1 18h
sh.helm.release.v1.redpanda.v32 helm.sh/release.v1 1 18h
sh.helm.release.v1.redpanda.v33 helm.sh/release.v1 1 18h
sh.helm.release.v1.redpanda.v34 helm.sh/release.v1 1 18h
sh.helm.release.v1.redpanda.v35 helm.sh/release.v1 1 17h
sh.helm.release.v1.redpanda.v36 helm.sh/release.v1 1 17h
sh.helm.release.v1.redpanda.v37 helm.sh/release.v1 1 17h
sh.helm.release.v1.redpanda.v38 helm.sh/release.v1 1 17h
sh.helm.release.v1.redpanda.v39 helm.sh/release.v1 1 16h
sh.helm.release.v1.redpanda.v40 helm.sh/release.v1 1 16h
sh.helm.release.v1.redpanda.v41 helm.sh/release.v1 1 16h
sh.helm.release.v1.redpanda.v42 helm.sh/release.v1 1 16h
sh.helm.release.v1.redpanda.v43 helm.sh/release.v1 1 15h
sh.helm.release.v1.redpanda.v44 helm.sh/release.v1 1 15h
sh.helm.release.v1.redpanda.v45 helm.sh/release.v1 1 15h
sh.helm.release.v1.redpanda.v46 helm.sh/release.v1 1 15h
sh.helm.release.v1.redpanda.v47 helm.sh/release.v1 1 14h
sh.helm.release.v1.redpanda.v48 helm.sh/release.v1 1 14h
sh.helm.release.v1.redpanda.v49 helm.sh/release.v1 1 14h
sh.helm.release.v1.redpanda.v50 helm.sh/release.v1 1 14h
sh.helm.release.v1.redpanda.v51 helm.sh/release.v1 1 13h
sh.helm.release.v1.redpanda.v52 helm.sh/release.v1 1 13h
sh.helm.release.v1.redpanda.v53 helm.sh/release.v1 1 13h
sh.helm.release.v1.redpanda.v54 helm.sh/release.v1 1 13h
sh.helm.release.v1.redpanda.v55 helm.sh/release.v1 1 12h
sh.helm.release.v1.redpanda.v56 helm.sh/release.v1 1 12h
sh.helm.release.v1.redpanda.v57 helm.sh/release.v1 1 12h
sh.helm.release.v1.redpanda.v58 helm.sh/release.v1 1 12h
sh.helm.release.v1.redpanda.v59 helm.sh/release.v1 1 11h
sh.helm.release.v1.redpanda.v60 helm.sh/release.v1 1 11h
sh.helm.release.v1.redpanda.v61 helm.sh/release.v1 1 11h
sh.helm.release.v1.redpanda.v62 helm.sh/release.v1 1 11h
sh.helm.release.v1.redpanda.v63 helm.sh/release.v1 1 10h
sh.helm.release.v1.redpanda.v64 helm.sh/release.v1 1 10h
sh.helm.release.v1.redpanda.v65 helm.sh/release.v1 1 10h
sh.helm.release.v1.redpanda.v66 helm.sh/release.v1 1 10h
sh.helm.release.v1.redpanda.v67 helm.sh/release.v1 1 9h
sh.helm.release.v1.redpanda.v68 helm.sh/release.v1 1 9h
sh.helm.release.v1.redpanda.v69 helm.sh/release.v1 1 9h
sh.helm.release.v1.redpanda.v70 helm.sh/release.v1 1 8h
sh.helm.release.v1.redpanda.v71 helm.sh/release.v1 1 8h
sh.helm.release.v1.redpanda.v72 helm.sh/release.v1 1 8h
sh.helm.release.v1.redpanda.v73 helm.sh/release.v1 1 8h
sh.helm.release.v1.redpanda.v74 helm.sh/release.v1 1 7h59m
sh.helm.release.v1.redpanda.v75 helm.sh/release.v1 1 7h43m
sh.helm.release.v1.redpanda.v76 helm.sh/release.v1 1 7h28m
sh.helm.release.v1.redpanda.v77 helm.sh/release.v1 1 7h13m
sh.helm.release.v1.redpanda.v78 helm.sh/release.v1 1 6h58m
sh.helm.release.v1.redpanda.v79 helm.sh/release.v1 1 6h42m
sh.helm.release.v1.redpanda.v80 helm.sh/release.v1 1 6h27m
sh.helm.release.v1.redpanda.v81 helm.sh/release.v1 1 6h12m
sh.helm.release.v1.redpanda.v82 helm.sh/release.v1 1 5h57m
sh.helm.release.v1.redpanda.v83 helm.sh/release.v1 1 5h41m
sh.helm.release.v1.redpanda.v84 helm.sh/release.v1 1 5h26m
sh.helm.release.v1.redpanda.v85 helm.sh/release.v1 1 5h11m
sh.helm.release.v1.redpanda.v86 helm.sh/release.v1 1 4h56m
sh.helm.release.v1.redpanda.v87 helm.sh/release.v1 1 4h40m
sh.helm.release.v1.redpanda.v88 helm.sh/release.v1 1 4h25m
sh.helm.release.v1.redpanda.v89 helm.sh/release.v1 1 4h10m
sh.helm.release.v1.redpanda.v90 helm.sh/release.v1 1 3h55m
sh.helm.release.v1.redpanda.v91 helm.sh/release.v1 1 3h39m
sh.helm.release.v1.redpanda.v92 helm.sh/release.v1 1 3h24m
sh.helm.release.v1.redpanda.v93 helm.sh/release.v1 1 3h9m
sh.helm.release.v1.redpanda.v94 helm.sh/release.v1 1 174m
sh.helm.release.v1.redpanda.v95 helm.sh/release.v1 1 158m
sh.helm.release.v1.redpanda.v96 helm.sh/release.v1 1 143m
sh.helm.release.v1.redpanda.v97 helm.sh/release.v1 1 128m
sh.helm.release.v1.redpanda.v98 helm.sh/release.v1 1 112m
sh.helm.release.v1.redpanda.v99 helm.sh/release.v1 1 97m
Steps to reproduce:
- Create a healthy setup
- Make the Statefulset fail scheduling new PODs, e.g. by
kubectl label namespace NAMESPACE pod-security.kubernetes.io/enforce=baseline - Delete the running PODs & either force rollout restart of the STS, or wait for the operator to do it.
JIRA Link: K8S-194
For reference https://kubernetes.io/docs/concepts/security/pod-security-standards/#baseline https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-namespace-labels/
Just for reference, the operator seems to have cleaned-up all the secrets when the workload got healthy again
Interesting. By when the workload got healthy again you mean that Redpanda resource has changed, so that Redpanda Pods become healthy?
By workload got healthy I mean the PODs became ready (as a result of switching to privileged PodSecurity policy for the given namespace).
Closing out due to the age of this issue. It seems like things were cleaned up by the operator as expected.
