Delete DBInstance fails with "FinalDBSnapshotIdentifier can not be specified"

[alexambarch]

What happened?

After creating an RDS cluster with AWS Native as part of a larger stack, I am now attempting to delete the resources so that I can recreate it all from scratch. Running pulumi destroy, pulumi destroy -t <db instance urn> --target-dependents, or trying to replace the resource at all results in the following error:

  aws-native:rds:DBInstance (rdsInstance):
    error: operation DELETE failed with "InvalidRequest": FinalDBSnapshotIdentifier can not be specified when deleting a cluster instance (Service: Rds, Status Code: 400, Request ID: a477dccc-baed-4cbc-94c0-878e5ed8f3af)

I have not specified any kind of final snapshot identifier in the configuration.

Expected Behavior

The resources are destroyed by running pulumi destroy or replaced when trying to replace the resource via replaceOnChanges.

Steps to reproduce

Configure a brand new pulumi stack as follows and run pulumi up, setting both aws:region and aws-native:region to us-east-1

// index.ts

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsn from "@pulumi/aws-native";

const vpc = new awsn.ec2.VPC("vpc", {
    cidrBlock: "10.0.0.0/16"
})

const subnet = new awsn.ec2.Subnet("subnet", {
    vpcId: vpc.id,
    cidrBlock: "10.0.1.0/24",
    availabilityZone: "us-east-1a"
})

const anotherSubnet = new awsn.ec2.Subnet("another-subnet", {
    vpcId: vpc.id,
    cidrBlock: "10.0.2.0/24",
    availabilityZone: "us-east-1b"
})

const subnetGroup = new aws.rds.SubnetGroup("rds-subnet-group", {
  subnetIds: [subnet.id, anotherSubnet.id]
});

const rdsCluster = new awsn.rds.DBCluster("rds-cluster", {
  dBClusterIdentifier: "clusterId",
  engine: aws.rds.AuroraPostgresqlEngine,
  engineVersion: "14.6",
  databaseName: "databaseName",
  masterUserPassword: "whatever",
  masterUsername: "username",
  dBSubnetGroupName: subnetGroup.name,
  serverlessV2ScalingConfiguration: {
    maxCapacity: 1,
    minCapacity: 0.5,
  }
});

const rdsInstance = new awsn.rds.DBInstance("rds-instance", {
  dBClusterIdentifier: rdsCluster.id,
  dBInstanceClass: "db.serverless",
  engine: aws.rds.AuroraPostgresqlEngine,
  engineVersion: "14.6"
});

After this finishes, attempt to destroy the stack by running pulumi destroy

Output of pulumi about

CLI
Version      3.61.0
Go Version   go1.20.2
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.35.0
aws-native  0.57.0
awsx        1.0.2
docker      3.6.1
nodejs      unknown

Host
OS       darwin
Version  13.3.1
Arch     arm64

This project is written in nodejs: executable='/Users/alexambarchian/.nvm/versions/node/v18.14.2/bin/node' version='v18.14.2'

Current Stack: aaambarc/bug-replication/dev

TYPE                             URN
pulumi:pulumi:Stack              urn:pulumi:dev::bug-replication::pulumi:pulumi:Stack::bug-replication-dev
pulumi:providers:aws-native      urn:pulumi:dev::bug-replication::pulumi:providers:aws-native::default_0_57_0
aws-native:ec2:VPC               urn:pulumi:dev::bug-replication::aws-native:ec2:VPC::vpc
aws-native:ec2:Subnet            urn:pulumi:dev::bug-replication::aws-native:ec2:Subnet::another-subnet
aws-native:ec2:Subnet            urn:pulumi:dev::bug-replication::aws-native:ec2:Subnet::subnet
pulumi:providers:aws             urn:pulumi:dev::bug-replication::pulumi:providers:aws::default_5_35_0
aws:rds/subnetGroup:SubnetGroup  urn:pulumi:dev::bug-replication::aws:rds/subnetGroup:SubnetGroup::rds-subnet-group
aws-native:rds:DBCluster         urn:pulumi:dev::bug-replication::aws-native:rds:DBCluster::rds-cluster


Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/aaambarc
User           aaambarc
Organizations  aaambarc, SyracuseU

Dependencies:
NAME                VERSION
@pulumi/aws-native  0.57.0
@pulumi/aws         5.35.0
@pulumi/awsx        1.0.2
@pulumi/pulumi      3.62.0
@types/node         16.18.23

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[kpitzen]

Hi @alexambarch - thank you for reporting this issue, and apologies for any inconvenience this has caused. This is likely due to SkipFinalSnapshot not being present in the CCAPI documentation. We'll almost certainly need to either implement that functionality in a custom manner on our end (which is not super desirable) or wait for it to be present in that documentation in a way that we can consume. As a temporary workaround, I notice that you have the AWS Classic provider available in your stack- I would suggest using rds.DBInstance there for the time being, and overriding the default skipFinalCheckpoint value to set it to true.

[duro]

Curious if there is any update here? Also, per the comment from @kpitzen, in order to use an instance with a Cluster, it would actually need to be an rds.ClusterInstance which also does not have the skipFinalCheckpoint property.

[RustyF]

I'd be happy to work on this one but is it actually waiting for a change to the cloud formation API?

[t0yv0]

Skimming through https://github.com/pulumi/pulumi-aws-native/blob/master/aws-cloudformation-schema/aws-rds-dbinstance.json it appears we're still waiting for a change in Cloud Control API, but please let us know if we're missing something obvious here. Thank you!

[t0yv0]

@flostadler suggests https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html is relevant here, checking up on it.

[t0yv0]

Regarding the AWSCloudFormation DeletionPolicy, looks like we are not sure yet how that maps to the Cloud Control API and hence this provider, unfortunately, will require some further research as time permits. Pointers appreciated!

[corymhall]

After looking into this a little further, I think it is a bug in the CCAPI. I've created an issue https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/2013