argoproj
Server-Side Apply prevents field deletion owned by other managers
Checklist:
- [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
- [x] I've included steps to reproduce the bug.
- [x] I've pasted the output of
argocd version.
Describe the bug
ArgoCD's server-side apply implementation cannot remove fields that were previously applied by client-side apply (kubectl apply) or other field managers. When migrating from client-side apply to server-side apply, removed fields remain in the resource indefinitely, even when they are no longer present in the desired manifest. This leads to resources that are inconsistent with their expected state.
The root cause is that server-side apply only manages fields that are explicitly specified in the applied manifest. Fields owned by other field managers (like kubectl with Update operation) persist even when they should be removed, because ArgoCD's field manager cannot take ownership of fields it doesn't explicitly apply.
This is the same issue described in https://github.com/kubernetes/kubernetes/issues/99003
As the upstream issue has still been open for awhile and doesn't look like it will be resolved soon, we should implement a workaround solution for this case as this will be a common case if users applied resources using kubectl apply before managing with argocd. Or if users have other legacy controllers managing fields that they want to later remove.
To Reproduce
- Apply a manifest using client-side apply (kubectl apply):
cat <<EOF | kubectl -n default apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: test-config
data:
key1: value1
key2: value2
legacy_field: should_be_removed
EOF
- Configure ArgoCD to manage this resource with server-side apply enabled. i.e add this file to Git and create an app that manages this file.
- Sync without ServerSideApply
- Observe in managed fields in argoCD. Since we applied using
kubectl apply -f,the fields are owned by `manager: kubectl-client-side-apply
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:key1: {}
f:key2: {}
f:legacy_field: {}
f:metadata:
f:annotations: {}
manager: kubectl-client-side-apply
operation: Update
time: '2025-05-30T20:29:26Z'
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:argocd.argoproj.io/tracking-id: {}
f:kubectl.kubernetes.io/last-applied-configuration: {}
manager: argocd-controller
operation: Update
time: '2025-05-30T20:30:09Z'
- Remove the field in Git
kind: ConfigMap
metadata:
name: test-config
data:
key1: value1
key2: value2
# legacy_field removed
- Sync with ServerSideApply
- Observe that the legacy_field still exists in the cluster:
apiVersion: v1
kind: ConfigMap
metadata:
name: test-config
data:
key1: value1
key2: value2
legacy_field: should_be_removed
Expected behavior
Field should be removed from live manifest after sync with ServerSideApply
Screenshots
Version
argocd version
argocd: v2.14.9+38985bd.dirty
BuildDate: 2025-04-02T23:02:54Z
GitCommit: 38985bdcd6c3b031fb83757a1fb0c39a55bf6a24
GitTreeState: dirty
GoVersion: go1.24.2
Compiler: gc
Platform: darwin/amd64
argocd-server: v3.0.3+c27a9d3.dirty
BuildDate: 2025-05-27T23:39:57Z
GitCommit: c27a9d33603b399b228d896e98d4a5fc86afd53c
GitTreeState: dirty
GoVersion: go1.24.3
Compiler: gc
Platform: darwin/amd64
Kustomize Version: v5.4.3 2024-07-19T16:40:33Z
Helm Version: v3.17.1+g980d8ac
Kubectl Version: v0.32.2
Jsonnet Version: v0.20.0
Logs
Paste any relevant application logs here.
FluxCD has run into this issue and fixed it by adding a 'Cleanup' applyOption that manipulates the managed fields and replaces problematic managers like kubectl- with their own manager https://github.com/fluxcd/kustomize-controller/pull/527
We could do something similar and add a cleanup option in conjunction with ServerSideApply to clean up managed fields before applying.
Similar to #22797
Would've been nice to have something this upstream (kubernetes/kubernetes#124191), but since that's not coming anytime soon I agree it should be implemented in Argo CD instead
