Deleting an ArgoCD Application from another ArgoCD App that manages it (App of Apps) leaves the child resources

[sidewinder12s]

If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a question in argocd slack channel.

Checklist:

• [x ] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [x ] I've included steps to reproduce the bug.
• [x ] I've pasted the output of argocd version.

Describe the bug

When using the App of Apps pattern, if you delete a child ArgoCD App from the parent and use prune, foreground delete, the application is deleted but the resources it was managing are left behind in the cluster.

To Reproduce

Deploy an App of Apps, remove/delete one of the child apps from the parent using the prune, foreground delete from within the parent app. Check for the app in ArgoCD (it should be gone), then check for the resources that the child app was managing (those still exist).

Expected behavior

I'd expect the child app and the resources it manages to be deleted according to the deletion policy selected.

Version

argocd-server: v2.0.4+0842d44
  BuildDate: 2021-06-23T01:27:53Z
  GitCommit: 0842d448107eb1397b251e63ec4d4bc1b4efdd6e
  GitTreeState: clean
  GoVersion: go1.16
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: v3.9.4 2021-02-09T19:22:10Z
  Helm Version: v3.5.1+g32c2223
  Kubectl Version: v0.20.4
  Jsonnet Version: v0.17.0

Logs

I hadn't found anything suspicious in our logs.

Paste any relevant application logs here.

[jaymebrd]

When using the App of Apps pattern, if you delete a child ArgoCD App from the parent and use prune, foreground delete, the application is deleted but the resources it was managing are left behind in the cluster.

To give another perspective this behaviour is something I suspect many people who are using the Apps of Apps pattern prefer because it significantly reduces the blast radius if human/automation errors were to delete a parent App (argoproj.io/Application) that manages an entire K8s cluster or a set of K8s clusters - and all those resources would get pruned. You could potentially take down all your production clusters by deleting a single Argo app.

At the very least, I think any change to this behaviour should require some configurable opt-in and be well documented.

[sidewinder12s]

When deleting it does offer the 3 normal ways of removing apps, foreground delete, background or orphan. The issue here is I leave it to the default foreground it doesn't actually do what I expect.

I also then don't really have a way to detect these orphaned resources since that feature is a project level feature and I'm deleting the project.

[prattyushmangal]

Hi all, I have also hit this problem on my Argo Deployment. I agree with @sidewinder12s that the default foreground delete of the Application does not clean up its child resources when the Application is managed by an AppSet, does not match my expectations.

Manual cleanup of the Orphaned resources is not a great experience either. So wanted to see if there were any updates on this ticket?

[lucasff]

I use it for preview environments and I would like to see some sort of cascading deletion. Using the finalizer actually gives me more trouble than the expected child deletion

[m44soroush]

We still have this problem, consider we have a parent argo application which has over 30 child argo applications with kind helm for each microservice. And If I want to change one of these application names, it would not delete previous resources, and new resources will be created, so there will be two set of resources with old and new name which will cause problems.

[rjeczkow]

Did you try to add this finalizer on your child apps : https://argo-cd.readthedocs.io/en/stable/user-guide/app_deletion/#about-the-deletion-finalizer https://argo-cd.readthedocs.io/en/stable/operator-manual/cluster-bootstrapping/#cascading-deletion

It worked for me in exactly the same scenario as yours.

[hsmade]

What the (big) downside of the current way of working is, is that resources become ghosts. If I for example change the values for an app-of-apps, which disables one of the apps in it, the app gets the delete but the resources stay. These resources still have the argo annotations/labels. These are now completely invisible for the (argo) user. And as the resources don't indicate that they're not managed anymore, it's impossible to detect this (to alert on it, for example, or to clean up). Even worse, if you change an app, for example you rename the resources in it, argo just creates new resources and leaves the old ones existing, unmanaged and hidden from the UI.

I can understand the reasoning to limit the blast radius. But how that is currently implemented might be even worse.