aws_rds: Add support for the Data API to DatabaseCluster construct

[Dilski]

Describe the feature

As of December 2023, the Data API is now supported for Aurora Serverless v2 and Aurora Provisioned database clusters (PostgreSQL for now) (Accouncement Blog).

Use Case

I want a straightforward way to enable and use Data API on my Aurora Serverless v2 cluster in the CDK

Proposed Solution

I think that the DatabaseCluster construct should get:

• enableDataApi construct prop
• grantDataApiAccess(grantee) method

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

2

Environment details (OS name and version, etc.)

n/a

[pahud]

Serverless v2 cluster is actually built on top of AWS::RDS::DBCluster resource but I didn't find any relevant property from the CFN doc. Maybe I was wrong but feel free to feedback if you find any document with that support.

[Dilski]

The CFN property is called enableHttpEndpointEnableDataApi is just the property name the Serverless v1 CDK construct uses

[neilferreira]

@Dilski @pahud I think there is going to be a bit more to this?

You are right, Serverless V2 is built on top of the AWS::RDS::DBCluster resource, but so is v1.

In order to use Aurora Serverless V2 you would need to specify https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#cfn-rds-dbcluster-serverlessv2scalingconfiguration

So I would imagine perhaps a ServerlessV2Cluster (similar to https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_rds.ServerlessCluster.html) but the scaling parameter would be populated with the configuration more relevant to the Serverless V2 scaling configuration instead. Alternatively, ServerlessCluster could accept a ServerlessVersion parameter, which dictates if ScalingConfiguration or ServerlessV2ScalingConfiguration is synthesized to the compiled CloudFormation template.

Someone would need to have a think about what this SDK might look like moving forward, and how to perhaps structure it in a way that doesn't impact anyone that uses Serverless V1 for now, but provides the relevant plumbing to create a Serverless V2 database moving forward.

[neilferreira]

Related https://github.com/aws/aws-cdk/pull/25437 (cc @corymhall)

[Dilski]

@neilferreira luckily it won't need to be as complex as that. Data API is supported on Aurora clusters now - both serverless and provisioned!

I imagine that all that is required is a constructor prop on the database cluster construct, and maybe a check in the constructor that the engine prop is aurora and throw an error otherwise

[neilferreira]

Does anyone have any useful/comprehensive resources on how to implement an Aurora Serverless v2 database cluster using CDK?

I've stumbled upon a few examples, but at a minimum, we still need to add a property override for the HTTP API, and rds.DatabaseCluster requires a writer to be specified, which from my understanding, is not a requirement when you intend to only use the HTTP API for your database.

[TheJoshuaEvans]

@neilferreira Here is an example setup. I also included a workaround for anybody else searching for the issue :)

const RDS = require('aws-cdk-lib/aws-rds');

const dbCluster = new RDS.DatabaseCluster(this, 'AuroraServerlessV2DatabaseCluster', {
  engine: RDS.DatabaseClusterEngine.auroraPostgres({
    version: RDS.AuroraPostgresEngineVersion.VER_15_3,
  }),
  serverlessV2MinCapacity: 0.5,
  serverlessV2MaxCapacity: 2,

  writer: RDS.ClusterInstance.serverlessV2('AuroraServerlessWriter'),
  readers: [
    RDS.ClusterInstance.serverlessV2('AuroraServerlessReader0', {scaleWithWriter: true}),
  ],
});

// Enable the data api via "layer 1" shenanigans
dbCluster.node.defaultChild.addOverride('Properties.EnableHttpEndpoint', true);

[csotomon]

@neilferreira here is our implementation using python @TheJoshuaEvans thank U for the instruction to enable de RDS Data API

def _create_rds(self, configurations: dict, rds_parameter_group: rds.ParameterGroup) -> rds.DatabaseCluster:
        rds_config = configurations['rds']
        

        ec2_vpc = ec2.Vpc.from_lookup(
            self, f"{configurations['projectName']}-{rds_config['name']}-vpc", vpc_id=rds_config['vpcId'])

        # Create the serverless cluster, provide all values needed to customise the database.
        subnets = [ec2.Subnet.from_subnet_id(self, f"{rds_config['name']}-subnet-{idx}", x)
                   for idx, x in enumerate(rds_config['subnetGroup']['subnetIds'])]
        subnet_group = rds.SubnetGroup(self, f"{configurations['projectName']}-{rds_config['name']}-SubnetGroup",
                                       description=f"{configurations['projectName']}-{rds_config['name']}-SubnetGroup",
                                       vpc=ec2_vpc,
                                       subnet_group_name=f"{configurations['projectName']}-{rds_config['name']}-SubnetGroup",
                                       vpc_subnets=ec2.SubnetSelection(
                                           subnets=subnets)
                                       )

        rds_security_group = ec2.SecurityGroup(self, f"{rds_config['name']}-sg",
                                               security_group_name=f"{configurations['projectName']}-{rds_config['name']}-sg",
                                               vpc=ec2_vpc,
                                               allow_all_outbound=True,
                                               disable_inline_rules=True
                                               )

        for ipv4_cidr_block in rds_config['securityGroup']['ingressRule']:
            rds_security_group.add_ingress_rule(
                peer=ec2.Peer.ipv4(ipv4_cidr_block),
                connection=ec2.Port.tcp(rds_config['port'])
            )
            
        rds_security_group.add_ingress_rule(
            ec2.Peer.security_group_id(rds_security_group.security_group_id),
            ec2.Port.tcp_range(0, 65535),
            "Allow intra-SG TCP traffic")
        
        rds_security_group.add_egress_rule(
            ec2.Peer.security_group_id(rds_security_group.security_group_id),
            ec2.Port.tcp_range(0, 65535),
            "Allow intra-SG TCP traffic")
        

        cluster = rds.DatabaseCluster(self,
                                      f"{configurations['projectName']}-{rds_config['name']}-cluster",
                                      default_database_name=rds_config['databaseName'],
                                      cluster_identifier=f"{configurations['projectName']}-{rds_config['name']}",
                                      engine=rds.DatabaseClusterEngine.aurora_postgres(
                                          version=rds.AuroraPostgresEngineVersion.VER_15_3),
                                      subnet_group=subnet_group,
                                      writer=rds.ClusterInstance.serverless_v2("writer",
                                                                               publicly_accessible=False,
                                                                               parameter_group= rds_parameter_group
                                                                               ),
                                      vpc_subnets=ec2.SubnetSelection(
                                          subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS),
                                      vpc=ec2_vpc,
                                      port=rds_config['port'],
                                      security_groups=[rds_security_group],
                                      deletion_protection=True,
                                      storage_encrypted=True,
                                      storage_type=rds.DBClusterStorageType.AURORA,
                                      storage_encryption_key=kms.Key(self, f"{rds_config['name']}-storageKey",
                                                                     alias=f"{configurations['projectName']}-{rds_config['name']}-storageKey",
                                                                     removal_policy=RemovalPolicy.DESTROY,
                                                                     enable_key_rotation=True),
                                      iam_authentication=True,
                                      serverless_v2_min_capacity=rds_config['capacity']['minimum'],
                                      serverless_v2_max_capacity=rds_config['capacity']['maximum']
                                      
                                      )

        # Enable RDS Data API
        cluster.node.default_child.add_override('Properties.EnableHttpEndpoint', True)
        
        return cluster

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.