[Bug][helm] VFN PVCs are not deleted after era bump

[rustielin]

🐛 Bug

To reproduce

Spin up a validator/VFN combo using aptos-node helm chart, and then bump the era, which replaces the underlying storage volumes. The validator PVs are deleted, but not the VFN PVs.

Check by running kubectl get pvc

This does not happen with validator PVs, likely because the PVCs are defined as a separate API object rather than in volumeClaimTemplates: https://github.com/aptos-labs/aptos-core/blob/1bb039eec615ad33d3949a597e72aa11d51b89a6/terraform/helm/aptos-node/templates/fullnode.yaml#L47

Expected Behavior

The PV/PVCs should be cleared after era wipe to save cost. Or have an option to explicitly keep them after era wipe.

System information

Check this across k8s in all supported clouds

[rustielin]

Seems related to https://kubernetes.io/blog/2021/12/16/kubernetes-1-23-statefulset-pvc-auto-deletion/, which explains that this is the previously intended behavior. Possible solutions to the above problem:

• Do not recommend bumping the helm/TF era variable (to trigger creation of new PVCs), and document it
• Manually delete the VFN PVCs after changing to a new era
• Upgrade to k8s v1.23 and use the PVC retention policy alpha feature

[github-actions[bot]]

This issue is stale because it has been open 45 days with no activity. Remove the stale label or comment - otherwise this will be closed in 15 days.

[github-actions[bot]]

This issue is stale because it has been open 45 days with no activity. Remove the stale label or comment - otherwise this will be closed in 15 days.

[rustielin]

Closing this, as it has been addressed by documentation