If composition and managed resource are created on top of an existing resource, they will not become "Ready"

[chobostar]

What happened?

Managed resource:

apiVersion: s3.aws.crossplane.io/v1beta1
kind: Bucket
metadata:
  annotations:
    crossplane.io/composition-resource-name: bucket
    crossplane.io/external-name: example-com-dev10-pg-backup
  creationTimestamp: "2023-05-23T10:25:27Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generateName: pg-backup-97hcs-
  generation: 5
  labels:
    crossplane.io/claim-name: pg-backup
    crossplane.io/claim-namespace: example-com-dev10
    crossplane.io/composite: pg-backup-97hcs
  name: example-com-dev10-pg-backup
  ownerReferences:
  - apiVersion: s3.aws.example.com/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: CompositeCrossplaneS3Bucket
    name: pg-backup-97hcs
    uid: 4d1722ce-1572-4ce0-9645-c9f05bf254df
  resourceVersion: "4005796615"
  uid: a38fbd68-91ed-4caf-ac24-b519f1ecead9
spec:
  deletionPolicy: Delete
  forProvider:
    accelerateConfiguration:
      status: Enabled
    acl: private
    locationConstraint: eu-central-1
    publicAccessBlockConfiguration:
      blockPublicAcls: true
      blockPublicPolicy: true
      ignorePublicAcls: true
      restrictPublicBuckets: true
    serverSideEncryptionConfiguration:
      rules:
      - applyServerSideEncryptionByDefault:
          sseAlgorithm: AES256
    tagging:
      tagSet:
      - key: owner
        value: example-com-dev10
      - key: environment
        value: cluster12
      - key: role
        value: client
      - key: crossplane-monitoring
        value: "false"
      - key: test-key
        value: test-value
  providerConfigRef:
    name: crossplane-aws
  writeConnectionSecretToRef:
    name: example-com-dev10-pg-backup-connection
    namespace: example-com-dev10
status:
  atProvider:
    arn: arn:aws:s3:::example-com-dev10-pg-backup
  conditions:
  - lastTransitionTime: "2023-05-23T10:25:27Z"
    reason: ReconcileSuccess
    status: "True"
    type: Synced

Bucket with name example-com-dev10-pg-backup exists in AWS.

$ kubectl get bucket.s3.aws.crossplane.io example-com-dev10-pg-backup
NAME                          READY   SYNCED   AGE
example-com-dev10-pg-backup           True     46m

^ I expect here Ready: True

Composition:

apiVersion: s3.aws.example.com/v1alpha1
kind: CompositeCrossplaneS3Bucket
metadata:
  creationTimestamp: "2023-05-23T10:25:27Z"
  finalizers:
  - composite.apiextensions.crossplane.io
  generateName: pg-backup-
  generation: 4
  labels:
    app.kubernetes.io/instance: s3
    crossplane.io/claim-name: pg-backup
    crossplane.io/claim-namespace: example-com-dev10
    crossplane.io/composite: pg-backup-97hcs    
    velero.io/backup-name: fullbackup-sched-20230522023057
    velero.io/restore-name: restore-example-com-dev10
  name: pg-backup-97hcs
  resourceVersion: "4005422986"
  uid: 4d1722ce-1572-4ce0-9645-c9f05bf254df
spec:
  claimRef:
    apiVersion: s3.aws.example.com/v1alpha1
    kind: CrossplaneS3Bucket
    name: pg-backup
    namespace: example-com-dev10
  compositionRef:
    name: compositecrossplanes3buckets.s3.aws.example.com
  compositionUpdatePolicy: Automatic
  parameters:
    accelerateConfiguration:
      status: Enabled
  resourceRefs:
  - apiVersion: s3.aws.crossplane.io/v1beta1
    kind: Bucket
    name: example-com-dev10-pg-backup
  - apiVersion: iam.aws.crossplane.io/v1beta1
    kind: User
    name: example-com-dev10-pg-backup-iamuser
  - apiVersion: iam.aws.crossplane.io/v1beta1
    kind: AccessKey
    name: example-com-dev10-pg-backup-iamuser-accesskey
  - apiVersion: s3.aws.crossplane.io/v1alpha3
    kind: BucketPolicy
    name: example-com-dev10-pg-backup-bucketpolicy
status:
  bucketArn: arn:aws:s3:::example-com-dev10-pg-backup
  bucketName: example-com-dev10-pg-backup
  conditions:
  - lastTransitionTime: "2023-05-23T10:25:27Z"
    reason: ReconcileSuccess
    status: "True"
    type: Synced
  - lastTransitionTime: "2023-05-23T10:25:27Z"
    reason: Creating
    status: "False"
    type: Ready

logs:

2023-05-23T11:12:22.705548399Z 1.6848403427054923e+09	DEBUG	provider-aws	Reconciling	{"controller": "managed/bucket.s3.aws.crossplane.io", "request": "/example-com-dev10-pg-backup"}
2023-05-23T11:12:22.954169926Z 1.6848403429541256e+09	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/bucket.s3.aws.crossplane.io", "request": "/example-com-dev10-pg-backup", "uid": "a38fbd68-91ed-4caf-ac24-b519f1ecead9", "version": "4005845255", "external-name": "example-com-dev10-pg-backup", "requeue-after": 1684840522.954124}
2023-05-23T11:12:22.954252285Z 1.6848403429542272e+09	DEBUG	events	Normal	{"object": {"kind":"Bucket","name":"example-com-dev10-pg-backup","uid":"a38fbd68-91ed-4caf-ac24-b519f1ecead9","apiVersion":"s3.aws.crossplane.io/v1beta1","resourceVersion":"4005845255"}, "reason": "UpdatedExternalResource", "message": "Successfully requested update of external resource"}

How can we reproduce it?

1. Create s3 bucket using Crossplane
2. Change deletionPolicy to Orphan in managed resource
3. Delete resources (claim -> composite -> managed)
4. Create again

What environment did it happen in?

Crossplane version: v1.10.1

• Kubernetes version (use kubectl version) v1.23.8
• Kubernetes distribution (e.g. Tectonic, GKE, OpenShift) OpenShift

[patelronak]

Just some data point on this. When I imported a s3 bucket and it had different configuration compared to composite I have seen this behavior. If you undo Change deletionPolicy to Orphan in managed resource it will become ready in my scenario. You should be able to make changes once it becomes ready.

[cvegagimenez]

Also have this issue and just want to add that by using the annotation crossplane.io/external-name results in the same behaviour. It is not possible to reconcile the object correctly.

[MisterMX]

I just tested the MR above with v0.43.0 and I can't reproduce this bug. Can you try and debug the provider yourselves?

[xoanmi]

We have the same issue. After reconciliation, s3 buckets are not getting ready:

➜ k get buckets.s3.aws.crossplane.io foo
NAME.                                           READY   SYNCED   AGE
my-super-own-bucket-foo                             True     12d

Status:
  At Provider:
    Arn:  arn:aws:s3:::my-super-own-bucket-foo
  Conditions:
    Last Transition Time:  2023-11-22T10:43:55Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced

We are running crossplane aws provider v0.44.2. No errors on logs:

2023-12-04T12:13:05.233Z	DEBUG	events	Successfully requested update of external resource	{"type": "Normal", "object": {"kind":"Bucket","name":"my-super-own-bucket-foo","uid":"f16bf1db-2da6-43ac-9c64-5cc7f3e6fad1","apiVersion":"s3.aws.crossplane.io/v1beta1","resourceVersion":"40851091"}, "reason": "UpdatedExternalResource"}
2023-12-04T12:14:05.242Z	DEBUG	provider-aws	Reconciling	{"controller": "managed/bucket.s3.aws.crossplane.io", "request": {"name":"my-super-own-bucket-foo"}}
2023-12-04T12:14:05.681Z	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/bucket.s3.aws.crossplane.io", "request": {"name":"my-super-own-bucket-foo"}, "uid": "f16bf1db-2da6-43ac-9c64-5cc7f3e6fad1", "version": "40851091", "external-name": "my-super-own-bucket-foo", "requeue-after": "2023-12-04T12:15:05.681Z"}

[MisterMX]

@xoanmi can you post the YAML of your bucket?

[xoanmi]

@xoanmi can you post the YAML of your bucket?

You can reproduce it with a very simple example:

1. Create the S3 with deletionPolicy=Orphan
2. Delete it from kubernetes
3. Create it again with exactly the same settings

[MisterMX]

@xoanmi I am unable to reproduce this with a standard bucket or the MR posted above. Can you give me the YAML that you are using?

[xoanmi]

@xoanmi I am unable to reproduce this with a standard bucket or the MR posted above. Can you give me the YAML that you are using?

apiVersion: s3.aws.crossplane.io/v1beta1
kind: Bucket
metadata:
  annotations:
    crossplane.io/external-name: my-own-s3-bucket
  name: my-own-s3-bucket
spec:
  deletionPolicy: Orphan
  forProvider:
    acl: private
    locationConstraint: eu-central-1
    objectOwnership: BucketOwnerPreferred
    paymentConfiguration:
      payer: BucketOwner
    versioningConfiguration:
      status: Enabled
  managementPolicies:
    - '*'

We are creating the MR from a composition. Could be this related to the issue?

[MisterMX]

We are creating the MR from a composition. Could be this related to the issue?

Probably, if you are overwriting fields in the composition.

[peresureda]

I have done some tests it's a problem in the reconciliation loop:

Steps to reproduce it:

• Step 1: Create a Bucket:

apiVersion: s3.aws.crossplane.io/v1beta1
kind: Bucket
metadata:
  name: test33
  annotations:
    crossplane.io/external-name: test33
spec:
  deletionPolicy: Orphan
  forProvider:
    acl: private
    objectOwnership: BucketOwnerPreferred
    locationConstraint: us-east-1
    paymentConfiguration:
      payer: BucketOwner
    versioningConfiguration:
      status: Enabled

• Step 2 ( Remove MR ): kubectl delete buckets.s3.aws.crossplane.io test33

• Step 3: Repeat step 1.

• Final Output:

But if you include the Encryption configuration, the bucket reconciles :

serverSideEncryptionConfiguration:
      rules:
        - applyServerSideEncryptionByDefault:
            sseAlgorithm: AES256 

[github-actions[bot]]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[cvegagimenez]

/fresh

This issue still happening