one-observability-demo
one-observability-demo copied to clipboard
CDK deploy is failing with TooManyRequestsException
Running cdk deploy --context admin_role=$EKS_ADMIN_ARN Services --context dashboard_role_arn=$CONSOLE_ROLE_ARN --require-approval never
fails with message:
Services: creating CloudFormation changeset...
6:36:00 PM | CREATE_FAILED | Custom::AWSCDK-EKS-KubernetesResource | loadBalancerServiceAccount1D19AD3A
Received response status [FAILED] from custom resource. Message returned: TooManyRequestsException: Rate Exceeded.
at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)
at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:49:8)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)
at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.
Seeing the same behavior - most probably caused by https://github.com/aws-samples/one-observability-demo/pull/176 as it appears Lambda is getting throttled.
Folks, this is an AWS internal situation where Isengard throttles resources (low containment score). Try again in an older Isengard account and you should be good. You could also test this on Event Engine. Closing this thread.