tofu-controller icon indicating copy to clipboard operation
tofu-controller copied to clipboard

Dependency deletions with destroy: true occur in the wrong order

Open darrylweaver opened this issue 2 years ago • 3 comments

tf-controller 0.16.0-rc.3

When you want to remove all the managed Terraform resources, you would run a Terraform destroy on all the resources.

If you have multiple Terraform CRs with dependencies set between them and you set: destroyResourcesOnDeletion: true then when you delete the Terraform CRs, tf-controller correctly deletes resources in the reverse order and blocks on a reverse dependency with the message: Deletion in progress, but blocked, please delete XXX to continue

However, if you set the value: destroy: true on all Terraform CRs instead of deleting them, then tf-controller tries to destroy the resources in the wrong order, i.e. the same ordering as when running an apply and not the reverse ordering required for a destroy.

For example, resulting in attempting to delete the VPC prior to deleting the resources inside that VPC, which blocks the deletion of the VPC and the destroy failing to complete.

darrylweaver avatar Sep 27 '23 17:09 darrylweaver

Thank you @darrylweaver

Currently we're still relying on K8s asynchronous reconciliation to do deletion. To make everything deleted in the right order, we would need to adopt an algorithm from, for example, StatefulSets.

An idea would be:

  • Compute DAG
  • Sort the DAG with topology sorting
  • Block deletion of all objects in the sorted resources
  • Start the deletion process from in the reverse order

chanwit avatar Sep 28 '23 05:09 chanwit

@madAndroid to try and reproduce. Q: is this a feature or a bug?

lasomethingsomething avatar Nov 03 '23 11:11 lasomethingsomething

Currently ignores the dependencies and the expected behavior is that it would not.

lasomethingsomething avatar Nov 06 '23 14:11 lasomethingsomething