popeye icon indicating copy to clipboard operation
popeye copied to clipboard

Unable to locate resource reference for PVC used by a cronjob

Open masterphenix opened this issue 3 years ago โ€ข 3 comments




Describe the bug Popeye does not seem to be able to locate PVC reference when used by a cronjob :

Unused PVC ?
  ยท monitoring/thanos-compact ....................................................๐Ÿ”Š
    ๐Ÿ”Š [POP-400] Used? Unable to locate resource reference.

Cronjob :

$ kubectl -n monitoring get cronjob thanos-compact
NAME                                   SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
thanos-compact                   0 9 * * *         False                0        3h19m                  210d

$ kubectl -n monitoring get cronjob thanos-compact -o yaml
[...]
            volumeMounts:
            - mountPath: /etc/config
              name: config-volume
            - mountPath: /data
              name: mnt
[...]
          volumes:
          - configMap:
              defaultMode: 420
              name: thanos-store
            name: config-volume
          - name: mnt
            persistentVolumeClaim:
              claimName: thanos-compact
[...]

To Reproduce Steps to reproduce the behavior:

  1. Create a cronjob that uses a PVC
  2. Run popeye

Expected behavior Popeye should be able to pickup the resource reference.

Versions (please complete the following information):

  • Popeye 0.9.2
  • K8s 1.19.0 (AKS)

masterphenix avatar Jun 10 '21 12:06 masterphenix

I don't believe this is an issue with Popeye but rather a behavior of PVs and PVCs; did you set a reclaimPolicy? If not, I believe it defaults to delete such that the PV is deleted once its use is completed.

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#delete

Could you verify/confirm using kubectl get pvc and kubectl get pv?

ebcarty avatar Jun 15 '21 03:06 ebcarty

I confirm reclaim policy is set to "Delete". However, the PVC is never deleted, thus the PV is not deleted either. Both objects are aged more than 200 days. The cronjob is running daily, To me, the PVC exists and is indeed used by the cronjob, it should not be deteced as unused.

$ kubectl get pvc thanos-compact
NAME                                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                   AGE
thanos-compact                         Bound    pvc-2093dc6d-cb05-4085-a1dc-d681c8f3d79f   64Gi       RWO            managed-premium-westeurope-1   214d

$ kubectl get pv pvc-2093dc6d-cb05-4085-a1dc-d681c8f3d79f 
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                            STORAGECLASS                   REASON   AGE
pvc-2093dc6d-cb05-4085-a1dc-d681c8f3d79f   64Gi       RWO            Delete           Bound    monitoring/thanos-compact-aks20-euw-infra-main   managed-premium-westeurope-1            214d

masterphenix avatar Jun 15 '21 08:06 masterphenix

@masterphenix @ebcarty Thank you both for your comments! Popeye does not currently scan job/cronjob so that dependency on the pv/pvc won't be picked up. They are on the radar and will be implemented next...

derailed avatar Jun 22 '21 13:06 derailed

Fixed v0.20.0

derailed avatar Feb 17 '24 22:02 derailed