Immutable fields cannot be late initialized

[knottnt]

Is your feature request related to a problem? When a field is marked as immutable in generator.yaml and is also late initialized the controller receives an error when trying to patch the resource manifest. This is relevant for fields that AWS can set default values for, but cannot be changed after initial creation of the AWS Resource. An example of this would DBCluster's AvailabilityZones as described in this issue.

Example Reproduction of the current behavior

1. In the RDS Controller repo set the DBCluster resource's AvailabilityZones field with is_immutable:true and late_initialize: {} and run code-generator to apply the changes.

resources:
   ...
   DBCluster:
   ...
      fields:
         AvailabilityZones:
           late_initialize: {}
           is_immutable: true

2. Run the RDS Controller and apply a DBCluster manifest with less than three AvailabilityZones set.

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBCluster
metadata:
  name: sample-aurora-cluster
  annotations:
    services.k8s.aws/deletion-policy: retain
spec:
  engine: aurora-mysql
  engineVersion: "8.0.mysql_aurora.3.08.1"
  dbClusterIdentifier: sample-aurora-cluster
  autoMinorVersionUpgrade: true
  backupRetentionPeriod: 7
  availabilityZones: # Only two AZs set
    - us-east-2b
    - us-east-2a
  storageEncrypted: true
  deletionProtection: true
  enableCloudwatchLogsExports:
    - audit
    - error
    - general
  copyTagsToSnapshot: true

4. In the controller logs see that the late initialization of the AvailabilityZones failed due to the cell validation check

{"level":"error","ts":"2025-06-03T10:39:55.166-0700","msg":"Reconciler error","controller":"dbcluster","controllerGroup":"rds.services.k8s.aws","controllerKind":"DBCluster","DBCluster":{"name":"sample-aurora-cluster","namespace":"default"},"namespace":"default","name":"sample-aurora-cluster","reconcileID":"27943400-8cbd-433f-b4a7-d25c1b3d7a20","error":"DBCluster.rds.services.k8s.aws \"sample-aurora-cluster\" is invalid: spec.availabilityZones: Invalid value: \"array\": Value is immutable once set","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/Users/tknott/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/Users/tknott/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/Users/tknott/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:255"}

Describe the solution you'd like One possibility would be to re-introduce the old immutability check implementation that set an ACK Advisory condition an immutable field was changed. Would need to consider how this works alongside the new cell validation based implementation though.

Describe alternatives you've considered Mark immutable fields as required. Although this doesn't fully solve the issue where the field value is a collection that can be partially set and then defaulted as is the case for DBCluster.AvailabilityZones.

[github-actions[bot]]

Hello @knottnt 👋 Thank you for opening an issue in ACK! A maintainer will triage this issue soon.

We encourage community contributions, so if you're interested in tackling this yourself or suggesting a solution, please check out our Contribution and Code of Conduct guidelines.

You can find more information about ACK on our website.

[a-hilaly]

thanks for raising this issue @knottnt. I think we need to clarify the semantics of immutability here since there seems to be a mismatch between AWS and Kubernetes concepts:

1. in k8s, once an immutable field is set through the API server (during creation) it cannot be modified
2. in AWS land, some fields can get defaulted from the service but become immutable after the initial creation

I think we need to relax the validation for "aws-level immutable" fields. We can allow these to be defaulted by AWS, then add a clear advisory message explaining that while AWS set the default, the field can't be changed afterward. Thoughts?

[knottnt]

I think that generally makes sense. Just for extra clarity there a some edge cases that we'll need to make sure our solution handles.

1. AWS may not return the default value in the Create Response. Instead we'll need to late initialize the value from a Read operation following the initial creation.
2. In the case of collections, such as DBCluster.AvailabilityZones, the field may be partially populated in the initial manifest and later need to be updated to include additions from AWS.