Aurora serverless v2, i/o-optimized

[OperationalFallacy]

Name of the resource

AWS::RDS::DBCluster

Resource name

No response

Description

AWS recently added new feature, not sure if this is already in CF

Other Details

No response

[masteinhauser]

Notably, the underlying RDS API and CloudFormation does partially-support the new StorageType, however subsequent updates to the deployed CloudFormation Stack fail unexpectedly.

CloudFormation Error from the underlying RDS API:

Resource handler returned message: "You can't modify the storage type of this DB cluster yet. Try again after 2023-06-17T16:49:18.757Z. (Service: Rds, Status Code: 400, Request ID: 1491c06b-d263-4814-bfaa-72f4f80878a9)" (RequestToken: 9f344f83-613f-9490-5e2d-33105330aafa, HandlerErrorCode: InvalidRequest)

Notably, the above error was after a CFN update setting the new StorageType and then a subsequent update for an unrelated ParameterGroup reference change.

Sample configuration:

  DBCluster:
    Type: AWS::RDS::DBCluster
    Properties:
      DBClusterIdentifier: cfn-io-optimized-storage
      EnableCloudwatchLogsExports:
        - postgresql
      Engine: aurora-postgresql
      EngineVersion: "15.2"
      Port: 5432  # required due to AWS defaulting to MySQL's 3306 for Aurora Provisioned Postgres
      ServerlessV2ScalingConfiguration:
        MinCapacity: '0.5'
        MaxCapacity: '128'
      StorageEncrypted: true
      StorageType: aurora-iopt1  # default is 'aurora' for Aurora Standard pricing.

[osdrv]

The io-optimized storage feature is globally available now. Though, there is a limitation @masteinhauser mentioned: a 30-day cool down period between storage type flips. The limitation is there by design. It was mentioned in the public release notes https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-aurora-i-o-optimized/.

You can switch existing database clusters to Aurora I/O-Optimized once every 30 days and switch back to Aurora Standard at any time.

Hope this makes sense. Please give us a heads up if any other questions remain.

[masteinhauser]

Hi @osdrv. Yes, the 30 Day change limitation is well documented and understood by me, as a human. Unfortunately, CloudFormation itself does not seem to understand this limitation.

Specifically, the scenario that I keep running into is:

• Once I have enabled StorageType: aurora-iopt1 via a CloudFormation stack update
• Subsequent updates without changing the StorageType value fail with the following error:

Resource handler returned message: "You can't modify the storage type of this DB cluster yet. Try again after 2023-06-17T16:49:18.757Z. (Service: Rds, Status Code: 400, Request ID: 1491c06b-d263-4814-bfaa-72f4f80878a9)" (RequestToken: 9f344f83-613f-9490-5e2d-33105330aafa, HandlerErrorCode: InvalidRequest)

To be clear: I am making no changes to the StorageType value across these subsequent stack updates. Due to this bug, my CloudFormation stacks are now unable to be managed with changes to the AWS::RDS::DBCluster more than once every 30 days.

[osdrv]

@masteinhauser I really appreciate the clarification. I didn't get it initially that there is no change to the storage type. This one is on us. We will update this ticket once it is fixed. Thanks again.

[hibhag]

The Aurora IO optimized feature support has been rolled out in May 2023.

[masteinhauser]

Hello @hibhag, unfortunately while your statement is true for when the feature was released my bug reported is still broken based on re-testing the scenario that I described above just now. CloudFormation updates are not possible on an RDS cluster with a recently changed Storage Type. It is very likely we will need to abandon use of I/O Optimized Storage because CloudFormation support does not function!

Please do not close bugs without test case verification.

[masteinhauser]

I am attaching a screenshot of my testing right now to try and further convince @osdrv and @hibhag this is still broken and warn other AWS CloudFormation customers

[hibhag]

@masteinhauser Got it, we will take a look for this specific scenario. Thanks for quick testing.

[danielxoi]

Hello! @hibhag will this change to open status as it's looked at?

[mjuszczak]

@osdrv and @hibhag, also curious as to whether this can be re-opened due to the outstanding bug?

[alexbarnes]

We are also seeing this same issue. Is there a plan to reopen this ticket to address the shortcomings of the implementation of this new feature?

[taxbit-xinghai]

we are also seeing the same issue when we did not even change the storage type

[danielxoi]

@hibhag any word if this will be re-opened?

[khebul]

@danielxoi I believe the issue is resolved now, and no-op changes to aurora IO optimized storage time no longer trigger mandatory cool down period