Creating applicationautoscaling.ScalingPolicy fails

[ixti]

What happened?

Attempt to create application autoscaling ScalingPolicy fails with:

error: creating resource: reading resource state: operation error CloudControl: GetResource, https response error StatusCode: 400, RequestID: ***, ResourceNotFoundException: AWS::ApplicationAutoScaling::ScalingPolicy Handler returned status FAILED: null (HandlerErrorCode: NotFound, RequestToken: ***)

To make it worse, resources is actually created. And subsequent attempt to update the stack fails with new errror:

error: creating resource: creating resource (await): operation CREATE failed with "AlreadyExists": Scaling Policy Already Exists

AWS does not have any UI to manage those, so after this I have to manually remove policies:

aws application-autoscaling describe-scaling-policies \
  --service-namespace dynamodb \
  | jq -r '
    .ScalingPolicies[]
    | "aws application-autoscaling delete-scaling-policy"
      + " --service-namespace dynamodb"
      + " --policy-name " + .PolicyName
      + " --resource-id " + .ResourceId
      + " --scalable-dimension " + .ScalableDimension
  ' \
  | bash -ex -

Example

const kclTable = new awsNative.dynamodb.Table("kcl-table", {
  attributeDefinitions:  [{ attributeName: "leaseKey", attributeType: "S" }],
  keySchema:             [{ attributeName: "leaseKey", keyType: "HASH" }],
  provisionedThroughput: { readCapacityUnits: 1, writeCapacityUnits: 1 },
  tableName:             `${pulumi.getProject()}-${pulumi.getStack()}-kcl`
});

const kclTableName = kclTable.tableName.apply((name) => name!);

const readTarget = new awsNative.applicationautoscaling.ScalableTarget(`kcl-table-read-target`, {
  maxCapacity: 50,
  minCapacity: 1,
  resourceId: pulumi.interpolate`table/${kclTableName}`,
  scalableDimension: "dynamodb:table:ReadCapacityUnits",
  serviceNamespace: "dynamodb",
});

const writeTarget = new awsNative.applicationautoscaling.ScalableTarget(`kcl-write-target`, {
  maxCapacity: 50,
  minCapacity: 1,
  resourceId: pulumi.interpolate`table/${kclTableName}`,
  scalableDimension: "dynamodb:table:WriteCapacityUnits",
  serviceNamespace: "dynamodb",
});

new awsNative.applicationautoscaling.ScalingPolicy(`kcl-table-read-policy`, {
  policyName: pulumi.interpolate`DynamoDBReadCapacityUtilization:${readTarget.resourceId}`,
  policyType: "TargetTrackingScaling",
  resourceId: readTarget.resourceId,
  scalableDimension: readTarget.scalableDimension,
  serviceNamespace: readTarget.serviceNamespace,
  targetTrackingScalingPolicyConfiguration: {
    predefinedMetricSpecification: { predefinedMetricType: "DynamoDBReadCapacityUtilization" },
    targetValue: 85,
    scaleInCooldown: 300,
    scaleOutCooldown: 30
  }
}, {
  dependsOn: [readTarget]
});

new awsNative.applicationautoscaling.ScalingPolicy(`kcl-table-write-policy`, {
  policyName: pulumi.interpolate`DynamoDBWriteCapacityUtilization:${writeTarget.resourceId}`,
  policyType: "TargetTrackingScaling",
  resourceId: writeTarget.resourceId,
  scalableDimension: writeTarget.scalableDimension,
  serviceNamespace: writeTarget.serviceNamespace,
  targetTrackingScalingPolicyConfiguration: {
    predefinedMetricSpecification: { predefinedMetricType: "DynamoDBWriteCapacityUtilization" },
    targetValue: 85,
    scaleInCooldown: 300,
    scaleOutCooldown: 30
  }
}, {
  dependsOn: [writeTarget],
});

Output of pulumi about

CLI          
Version      3.124.0
Go Version   go1.22.5
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  aws         6.44.0
resource  aws-native  0.111.0
language  nodejs      unknown
resource  random      4.16.3

Host     
OS       gentoo
Version  2.15
Arch     x86_64

Backend        
Name           evil-eurasier
URL            s3://***
User           ixti
Organizations  
Token type     personal

Dependencies:
NAME                VERSION
nunjucks            3.2.4
@pulumi/aws-native  0.111.0
@pulumi/aws         6.44.0
@pulumi/pulumi      3.124.0
@pulumi/random      4.16.3
@types/node         20.14.10
@types/nunjucks     3.2.6

Additional context

The same works perfectly fine with AWS classic provider:

new aws.appautoscaling.Policy(`kcl-table-read-policy`, {
  name: pulumi.interpolate`DynamoDBReadCapacityUtilization:${readTarget.resourceId}`,
  policyType: "TargetTrackingScaling",
  resourceId: readTarget.resourceId,
  scalableDimension: readTarget.scalableDimension,
  serviceNamespace: readTarget.serviceNamespace,
  targetTrackingScalingPolicyConfiguration: {
    predefinedMetricSpecification: { predefinedMetricType: "DynamoDBReadCapacityUtilization" },
    targetValue: 85,
    scaleInCooldown: 300,
    scaleOutCooldown: 30
  }
}, {
  dependsOn: [readTarget]
});

new aws.appautoscaling.Policy(`kcl-table-write-policy`, {
  name: pulumi.interpolate`DynamoDBWriteCapacityUtilization:${writeTarget.resourceId}`,
  policyType: "TargetTrackingScaling",
  resourceId: writeTarget.resourceId,
  scalableDimension: writeTarget.scalableDimension,
  serviceNamespace: writeTarget.serviceNamespace,
  targetTrackingScalingPolicyConfiguration: {
    predefinedMetricSpecification: { predefinedMetricType: "DynamoDBWriteCapacityUtilization" },
    targetValue: 85,
    scaleInCooldown: 300,
    scaleOutCooldown: 30
  }
}, {
  dependsOn: [writeTarget],
});

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[corymhall]

@ixti thanks for reporting this issue! I've been able to reproduce the error and it looks like it is an issue in the CloudControl API. I've created an issue in the aws-cloudformation-roadmap repo so we'll see what they have to say.