EKS Cluster does not refresh cleanly

[t0yv0]

What happened?

Unexpected update plan for a refresh right after pulumi up:

View in Browser (Ctrl+O): https://app.pulumi.com/t0yv0/resources/eksdebug/updates/2

     Type                             Name                    Status           Info
     pulumi:pulumi:Stack              resources-eksdebug                       
     ├─ pulumi:providers:aws          provider                                 
 ~   ├─ aws:iam:Role                  ekscluster1role         updated (1s)     [diff: ~managedPolicyArns]
     ├─ aws:iam:RolePolicyAttachment  ekscluster1roleattach1                   
     ├─ aws:iam:RolePolicyAttachment  ekscluster1roleattach2                   
     ├─ aws:ec2:Vpc                   vpc1                                     
     ├─ aws:ec2:Subnet                subnet1                                  
     ├─ aws:ec2:Subnet                subnet2                                  
 ~   └─ aws:eks:Cluster               ekscluster1             updated (1s)     [diff: +defaultAddonsToRemoves]


Resources:
    ~ 2 updated
    7 unchanged

Duration: 3s

Expected Behavior

Refreshing eks.Cluster that was just provisioned does not generate update plans.

Steps to reproduce

name: resources

runtime:
  name: yaml

resources:

  provider:
    type: pulumi:providers:aws
    options:
      version: 5.42.0

  "ekscluster1":
    type: aws:eks:Cluster
    properties:
      roleArn: ${ekscluster1role.arn}
      vpcConfig:
        subnetIds:
          - ${subnet1.id}
          - ${subnet2.id}
    options:
      provider: ${provider}

  "ekscluster1role":
    type: aws:iam:Role
    properties:
      assumeRolePolicy:
        fn::invoke:
           return: json
           function: aws:iam:getPolicyDocument
           arguments:
             statements:
               - effect: Allow
                 principals:
                   - type: Service
                     identifiers:
                       - eks.amazonaws.com
                 actions:
                   - sts:AssumeRole
    options:
      provider: ${provider}

  "ekscluster1roleattach1":
    type: aws:iam:RolePolicyAttachment
    properties:
      policyArn: arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
      role: ${ekscluster1role.name}
    options:
      provider: ${provider}

  "ekscluster1roleattach2":
    type: aws:iam:RolePolicyAttachment
    properties:
      policyArn: arn:aws:iam::aws:policy/AmazonEKSVPCResourceController
      role: ${ekscluster1role.name}
    options:
      provider: ${provider}

  "vpc1":
    type: aws:ec2/vpc:Vpc
    properties:
      cidrBlock: "10.192.0.0/16"
    options:
      provider: ${provider}

  "subnet1":
    type: aws:ec2/subnet:Subnet
    properties:
      cidrBlock: 10.192.20.0/24
      vpcId: ${vpc1.id}
      mapPublicIpOnLaunch: false
      availabilityZone: us-west-2a
    options:
      provider: ${provider}

  "subnet2":
    type: aws:ec2/subnet:Subnet
    properties:
      cidrBlock: 10.192.21.0/24
      vpcId: ${vpc1.id}
      mapPublicIpOnLaunch: false
      availabilityZone: us-west-2b
    options:
      provider: ${provider}

Steps:

$ pulumi up --yes
$ pulumi refresh

Output of pulumi about

CLI          
Version      3.74.0
Go Version   go1.20.5
Go Compiler  gc

Plugins
NAME  VERSION
aws   5.42.0
yaml  unknown

Host     
OS       darwin
Version  13.4.1
Arch     x86_64

This project is written in yaml

Current Stack: t0yv0/resources/eksdebug

TYPE                                               URN
pulumi:pulumi:Stack                                urn:pulumi:eksdebug::resources::pulumi:pulumi:Stack::resources-eksdebug
pulumi:providers:aws                               urn:pulumi:eksdebug::resources::pulumi:providers:aws::provider
pulumi:providers:aws                               urn:pulumi:eksdebug::resources::pulumi:providers:aws::default
aws:iam/role:Role                                  urn:pulumi:eksdebug::resources::aws:iam/role:Role::ekscluster1role
aws:iam/rolePolicyAttachment:RolePolicyAttachment  urn:pulumi:eksdebug::resources::aws:iam/rolePolicyAttachment:RolePolicyAttachment::ekscluster1roleattach2
aws:iam/rolePolicyAttachment:RolePolicyAttachment  urn:pulumi:eksdebug::resources::aws:iam/rolePolicyAttachment:RolePolicyAttachment::ekscluster1roleattach1
aws:ec2/vpc:Vpc                                    urn:pulumi:eksdebug::resources::aws:ec2/vpc:Vpc::vpc1
aws:ec2/subnet:Subnet                              urn:pulumi:eksdebug::resources::aws:ec2/subnet:Subnet::subnet1
aws:ec2/subnet:Subnet                              urn:pulumi:eksdebug::resources::aws:ec2/subnet:Subnet::subnet2
aws:eks/cluster:Cluster                            urn:pulumi:eksdebug::resources::aws:eks/cluster:Cluster::ekscluster1


Found no pending operations associated with eksdebug

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/t0yv0
User           t0yv0
Organizations  t0yv0, pulumi

No dependencies found

Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/ by default

Additional context

This issue was discovered as part of building out provider upgrade tests and surfaced when recording baseline provider behavior for the eks.Cluster resource.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

On V6 this stack has a similar but different problem:

Do you want to perform this refresh?
No resources will be modified as part of this refresh; just your stack's state will be.
 details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:eksdebug::resources::pulumi:pulumi:Stack::resources-eksdebug]
    ~ aws:iam/role:Role: (update)
        [id=ekscluster1role-3d947ce]
        [urn=urn:pulumi:eksdebug::resources::aws:iam/role:Role::ekscluster1role]
        [provider=urn:pulumi:eksdebug::resources::pulumi:providers:aws::provider::72b09d72-d11c-41e8-9c98-b0557ce66591]
        --outputs:--
      - roleLastUseds      : [
      -     [0]: {
              - lastUsedDate: ""
              - region      : ""
            }
        ]
    ~ aws:ec2/vpc:Vpc: (update)
        [id=vpc-0d04740861cf1d0f6]
        [urn=urn:pulumi:eksdebug::resources::aws:ec2/vpc:Vpc::vpc1]
        [provider=urn:pulumi:eksdebug::resources::pulumi:providers:aws::provider::72b09d72-d11c-41e8-9c98-b0557ce66591]
        --outputs:--
      - enableClassiclink               : false
      - enableClassiclinkDnsSupport     : false

Do you want to perform this refresh?
No resources will be modified as part of this refresh; just your stack's state will be.
  [Use arrows to move, type to filter]
  yes

Using

  provider:
    type: pulumi:providers:aws
    options:
      version: 6.0.2-alpha.2+9b14ba69

[t0yv0]

Still a problem:

Previewing refresh (repro2)

View in Browser (Ctrl+O): https://app.pulumi.com/anton-pulumi-corp/resources/repro2/previews/22ff7902-cbf3-4dbd-916d-cd889bc19484

     Type                             Name                    Plan       Info
     pulumi:pulumi:Stack              resources-repro2                   
     ├─ pulumi:providers:aws          provider                           
 ~   ├─ aws:iam:Role                  ekscluster1role         update     [diff: ~managedPolicyArns]
     ├─ aws:iam:RolePolicyAttachment  ekscluster1roleattach2             
     ├─ aws:ec2:Subnet                subnet1                            
     ├─ aws:iam:RolePolicyAttachment  ekscluster1roleattach1             
     ├─ aws:ec2:Subnet                subnet2                            
     ├─ aws:ec2:Vpc                   vpc1                               
 ~   └─ aws:eks:Cluster               ekscluster1             update     [diff: +defaultAddonsToRemoves]

Resources:
    ~ 2 to update
    7 unchanged

Do you want to perform this refresh?
No resources will be modified as part of this refresh; just your stack's state will be.
 details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:repro2::resources::pulumi:pulumi:Stack::resources-repro2]
    ~ aws:iam/role:Role: (update)
        [id=ekscluster1role-1c24246]
        [urn=urn:pulumi:repro2::resources::aws:iam/role:Role::ekscluster1role]
        [provider=urn:pulumi:repro2::resources::pulumi:providers:aws::provider::9ac84da1-b67e-4670-856e-1ad436474fac]
        --outputs:--
      ~ managedPolicyArns  : [
          + [0]: "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
          + [1]: "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
        ]
    ~ aws:eks/cluster:Cluster: (update)
        [id=ekscluster1-421a7c7]
        [urn=urn:pulumi:repro2::resources::aws:eks/cluster:Cluster::ekscluster1]
        [provider=urn:pulumi:repro2::resources::pulumi:providers:aws::provider::9ac84da1-b67e-4670-856e-1ad436474fac]
        --outputs:--
      + defaultAddonsToRemoves : []

The problem is self-correcting after refresh. Looks like Read is normalizing defaultAddonsToRemoves from missing to [].

"default_addons_to_remove": {
				Type:     schema.TypeList,
				Optional: true,
				Elem: &schema.Schema{
					Type: schema.TypeString,
				},
			}

managedPolicyArns is exactly the same as in https://github.com/pulumi/pulumi-aws/issues/2246