kustomize-controller icon indicating copy to clipboard operation
kustomize-controller copied to clipboard

Kustomize Controller deletes resource when Kustomization encounters resource with invalid / nonsense spec

Open CAR6807 opened this issue 3 years ago • 2 comments

A resource is deleted when the resource is updated with an invalid spec.

Expected behavior is the resource fails to update but not delete the existing resource.

To reproduce

Prior to change, verify resource exist

kubectl get svc -n zebrafinch zt-ms-zf-master-pre-release
NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
zt-ms-zf-master-pre-release   ClusterIP   192.168.122.47   <none>        443/TCP   24s

Then introduce invalid spec change, in this case an empty ports section for the Service Object

---
apiVersion: v1
kind: Service
metadata:
  labels:
    contextRoot: zt-ms-zf
  name: ingress-zt-ms-zf-master-pre-release
  namespace: zebrafinch
spec:
  ports:  #<-------- note the ports section is empty
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
  type: ClusterIP

Once committed, the resource is deleted from the cluster even though the resource still exists in git and the update fails.

kubectl get svc -n zebrafinch zt-ms-zf-master-pre-release 
Error from server (NotFound): services "zt-ms-zf-master-pre-release " not found

Event from the Kustomize Object states it cannot find the object because it deleted by Kustomize

caas-zerotouch-ms-zebrafinch-master            396d   False     Service/zebrafinch/zt-ms-zf-master-pre-release immutable field detected, failed to delete object, error: services "zt-ms-zf-master-pre-release" not found

Spec from Kustomize Object include Prune

spec:
  decryption:
    provider: sops
  force: true
  interval: 1m0s
  path: ./caas/zerotouch-ms-zebrafinch/master
  prune: true
  serviceAccountName: zebrafinch
  sourceRef:
    kind: GitRepository
    name: zebrafinch
    namespace: zebrafinch
  targetNamespace: zebrafinch

Kustomize object still has resource in inventory

  inventory:
    entries:
     - id: zebrafinch_zt-ms-zf-master-pre-release__Service
      v: v1

Kustomize Controller should not delete resource if it encounters invalid spec.

CAR6807 avatar Sep 28 '22 17:09 CAR6807

It deletes the object because of force: true, the Kubernetes API returns the same error as for immutable changes.

stefanprodan avatar Sep 28 '22 17:09 stefanprodan

Odd thing I have seen this on CRDs where fields are not set as Immutable and otherwise can be updated manually with the invalid fields dropped

CAR6807 avatar Sep 28 '22 19:09 CAR6807