cloudformation-coverage-roadmap
cloudformation-coverage-roadmap copied to clipboard
aws-cloudformation
AWS::RDS::DBInstance - DbiResourceId accessible via Fn::GetAtt
Instructions for CloudFormation Coverage New Issues Template
- Title -> AWS::RDS::DBInstance-DbiResourceId accessible via Fn::GetAtt
- Scope of request -> Make the DbiResourceId accessible using Fn::GetAtt on an AWS::RDS::DBInstance resource.
- Expected behavior -> Get back the DbiResourceId.
- Test case recommendation (optional) ->
- Links to existing API doc (optional) ->
- Category tag (optional) -> Database
- Any additional context (optional)
The RDS instances now supports IAM authentication as described here. Unfortunately in the PolicyDocument you have to use the DbiResourceId as the reference to the target RDS instance. This is not known before and you can't query it from the CFn either. Currently I have to create the stack, fetch the DbiResourceId and then create a changeset to modify the Policy. It would so much simpler if we can do that in one step.
"DBInstance": {
"Type": "AWS::RDS::DBInstance",
"Properties": { }
},
"DBUserPolicy": {
"Type": "AWS::IAM::ManagedPolicy",
"Properties": {
"Description": "blah blah not relevant",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [ "rds-db:connect" ],
"Effect": "Allow",
"Resource": [
{ "Fn::Sub": "arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:db-KOUCHAID5AIS8GAN6XEIKOOTIZ/db_user_name" }
]
}
]
}
}
}
}
I would like to change the Resource entry like this:
"Fn::Sub": [
"arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:${DbiResourceId}/db_user_name",
{ "DbiResourceId": { "Fn::GetAtt": [ "DbInstance", "DbiResourceId" ] } } ]
This is a daft omission. The fact that you can enable IAM authentication via CFN but can't get the resource ID to setup an IAM policy for it is absurd. Why was this not included in the CFN update providing for enabling IAM authentication in the first place?
Come on. This was raised nearly 10 months ago. We're not asking for much...
Its really absurd, this issue is nearly 10 months old and nobody seems to care about this issue. The feature is completely useless i am not able to fetch the db-resource-id inside the cloudformation template.
Totally agree, it is absurd. I meditated on the architecture for a project for two days with IAM authorization in mind to discover, that can't automate policy creation with CF. Several workarounds are definitely possible here, but it will inevitably break "encapsulation". Hope this will be implemented soon.
DbiResourceId should have been available from the start. I keep coming back to this Github issue every few months when I have the same headache for a different project.. Strong +1 for this to be added to enhancement!
Here's a snippet from our template to lookup the DbiResourceId via a CustomResource:
"DBResourceId": {
"Properties": {
"DBInstanceIdentifier": {
"Ref": "Instance"
},
"ServiceToken": {
"Fn::GetAtt": [
"DBResourceIdLookupLambda",
"Arn"
]
}
},
"Type": "AWS::CloudFormation::CustomResource"
},
"DBResourceIdLookupLambda": {
"Properties": {
"Code": {
"ZipFile": "import logging\nimport boto3\nimport cfnresponse\n\nlog = logging.getLogger(__name__)\nlog.setLevel(logging.INFO)\n\n\ndef db_resource_name(instance_identifier: str):\n rds = boto3.client(\"rds\")\n return rds.describe_db_instances(DBInstanceIdentifier=instance_identifier)[\n \"DBInstances\"\n ][0][\"DbiResourceId\"]\n\n\ndef lambda_handler(event: dict, context) -> None:\n physical_resource_id = None\n if \"DBInstanceIdentifier\" not in event[\"ResourceProperties\"]:\n log.error(\"DBInstanceIdentifier not provided in ResourceProperties\")\n status = cfnresponse.FAILED\n elif event[\"RequestType\"] == \"Delete\":\n log.info(\"Stack is deleting. Skipping ID lookup.\")\n status = cfnresponse.SUCCESS\n else:\n instance_identifier = event[\"ResourceProperties\"][\"DBInstanceIdentifier\"]\n try:\n physical_resource_id = db_resource_name(instance_identifier)\n status = cfnresponse.SUCCESS\n except Exception:\n log.error(\n \"Error looking up DB instance identifier for '%s'\",\n instance_identifier,\n exc_info=True,\n )\n status = cfnresponse.FAILED\n\n cfnresponse.send(event, context, status, {}, physical_resource_id)\n"
},
"Handler": "index.lambda_handler",
"Role": {
"Fn::GetAtt": [
"DBResourceIdLookupRole",
"Arn"
]
},
"Runtime": "python3.7",
"Timeout": 10
},
"Type": "AWS::Lambda::Function"
},
"DBResourceIdLookupRole": {
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Sid": ""
}
],
"Version": "2012-10-17"
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
],
"Policies": [
{
"PolicyDocument": {
"Statement": [
{
"Action": "rds:DescribeDBInstances",
"Effect": "Allow",
"Resource": [
{
"Fn::Sub": "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${Instance}"
},
{
"Fn::If": [
"IsMultiAz",
{
"Fn::Sub": "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${ReplicaInstance}"
},
{
"Ref": "AWS::NoValue"
}
]
}
],
"Sid": ""
}
],
"Version": "2012-10-17"
},
"PolicyName": {
"Fn::Sub": "describe-db-instance"
}
}
]
},
"Type": "AWS::IAM::Role"
},
"DBResourceIdReplica": {
"Condition": "IsMultiAz",
"Properties": {
"DBInstanceIdentifier": {
"Ref": "ReplicaInstance"
},
"ServiceToken": {
"Fn::GetAtt": [
"DBResourceIdLookupLambda",
"Arn"
]
}
},
"Type": "AWS::CloudFormation::CustomResource"
}
The actual code of the lambda is:
import logging
import boto3
import cfnresponse
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
def db_resource_name(instance_identifier: str) -> str:
rds = boto3.client("rds")
return rds.describe_db_instances(DBInstanceIdentifier=instance_identifier)[
"DBInstances"
][0]["DbiResourceId"]
def lambda_handler(event: dict, context) -> None:
physical_resource_id = None
if "DBInstanceIdentifier" not in event["ResourceProperties"]:
log.error("DBInstanceIdentifier not provided in ResourceProperties")
status = cfnresponse.FAILED
elif event["RequestType"] == "Delete":
log.info("Stack is deleting. Skipping ID lookup.")
status = cfnresponse.SUCCESS
else:
instance_identifier = event["ResourceProperties"]["DBInstanceIdentifier"]
try:
physical_resource_id = db_resource_name(instance_identifier)
status = cfnresponse.SUCCESS
except Exception:
log.error(
"Error looking up DB instance identifier for '%s'",
instance_identifier,
exc_info=True,
)
status = cfnresponse.FAILED
cfnresponse.send(event, context, status, {}, physical_resource_id)
@newtondev @andrius-paurys @SamMcFaddenBJSS
If you react with the 👍 button to the original issue, (the first comment, click on the smiley face if you're the first reacting), your votes can be used to sort issues and determine priorities.
A comment will send a notification to everyone (participants and watchers), but cannot be easily counted as a vote for an issue. Thus It's generally better to vote than to comment with "+1". To keep up to date, you can also add yourself as a watcher.
The issue with this identifier can be worked around by retrieving it using the CLI for example. (I needed this to be able to create the IAM policy from CI/CD pipeline.)
DB_RESOURCE_ID=$(aws rds describe-db-instances --db-instance-identifier <ID> --query 'DBInstances[0].DbiResourceId' --output text)
Then DB_RESOURCE_ID can be passed as parameter to Cloudformation.
The bigger problem is you still need to configure the database (at least for MySQL and Postgres) to enable IAM based authentication (and to do so you need the password). So this feature would need both
- ability to get the DbiResourceId in CF
- ability to create database user with the right grant
To make the setup fully Cloudformation based.
TL;DR AWS have to expose the DbiResourceId as they already did for Endpoint.Address and Endpoint.Port
In the meantime (because a 2y+ opened issue is quite a long time to adopt a wait&see approach) you can follow one of this well described work around https://aws.amazon.com/blogs/mt/four-ways-to-retrieve-any-aws-service-property-using-aws-cloudformation-part-3-of-3/
It's not ideal but at least you can retrieve and use any attribute that Cloudformation don't expose.
hey folks, thanks for your patience on this. this is coming soon. stay tuned!
@TheDanBlanco thanks for the feature! Is there any chance we could get a similar functionality from AWS::RDS::DBProxy? It also has a ResourceId associated to it that can't be output as of right now
