aws-samples
[deployment fails]
On a clean account, deployment fails, with the Services CloudFormation stack failing on on resource logical ID: loadBalancerServiceAccount1D19AD3A
Received response status [FAILED] from custom resource. Message returned: TimeoutError: {"state":"TIMEOUT","reason":"Waiter has timed out"} at checkExceptions (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/util-waiter/dist-cjs/index.js:59:26) at waitUntilFunctionActiveV2 (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:5820:49) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async defaultInvokeFunction (/var/task/outbound.js:1:1024) at async invokeUserFunction (/var/task/framework.js:1:2794) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1837) (RequestId: b7764054-fe42-40fc-bf8e-8096cbde4b65)
Hello @psantus, Thank you for your patience. I have recently on-boarded to the team and I'm following up with issues for this project. I was able to successfully deploy the current version of the code.
Could you confirm if you're still seeing this issue? I may need more details if so.
Could you do that on a new account? Now new accounts have default quotas lower than so called default quotas.
@AWS-Joe-A: adding a list of all the quotas request we need to make to get this deployment to work would be a satisfactory solution to this issue.
Hello @psantus, Thank you for the feedback. I'm deploying a new account and will test again. I'll let you know what I find. Thank you!
Hello @psantus, I created a new account and was able to successfully deploy the one-observability-demo. I ran through a few modules without issue.
Did any of the other stacks or nested stack have errors as well?
Issue was in one nested stack. I deployed in eu-west-1.
I'll try again and will let you know
I deployed in us-east-1 as I didn't se a region listed in the error message. I will also attempt to deploy it in eu-west-1 now .
Hello @psantus, I was able to deploy in eu-west-1 successfully in the new account as well. Please let me know what you find as well. Ty!
Hi @AWS-Joe-A
I appreciate your commitment and efforts to replicate. I just deployed in my account again and got 2 errors
in the Services stack
{"state":"TIMEOUT","reason":"Waiter has timed out"} at checkExceptions (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/util-waiter/dist-cjs/index.js:59:26) at waitUntilFunctionActiveV2 (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:5820:49) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async defaultInvokeFunction (/var/task/outbound.js:1:1024) at async invokeUserFunction (/var/task/framework.js:1:2794) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1837) (RequestId: 7f698e42-ca06-48d6-b2b9-a5a3b5ddf9a0)
in the Observability-Workshop stack
WaitCondition received failed message: 'CodePipeline Deploy ended' for uniqueId: abd96809-8b7e-42fa-967f-d308a96f6f9b
@AWS-Joe-A what's your timezone? I can be available today from 3PM to 5PM UTC if you want to have a look together on my deployment
Hello @psantus, Thank you for your patience. I'm sorry we weren't able to connect earlier. Let me reach out internally to see if I can identify someone closer to UTC time zone to work with you on this. It may be difficult to find someone due to the holidays, but I'll do my best to find someone that could help.
I'm was seeing a similar failure attempting to deploy this workshop via cloudshell: WaitCondition received failed message: 'CodePipeline Deploy ended' for uniqueId: 22342e88-f6d1-4692-ad1e-a56b73c2a2fe
I noticed the CDK deployment had failed as well, looking deeper, I noticed the rollback had left a bunch of buckets behind, and this was causing a failure on the CreateRepository step of the CDK deploy, I manually emptied and deleted the buckets and the deployment was successful.
Lucky you.
I did the same cleaning with no luck. Also tried on two different accounts.
On Mon, Jan 6, 2025 at 11:00 PM grhartt-aws @.***> wrote:
I'm was seeing a similar failure attempting to deploy this workshop via cloudshell: WaitCondition received failed message: 'CodePipeline Deploy ended' for uniqueId: 22342e88-f6d1-4692-ad1e-a56b73c2a2fe
image.png (view on web) https://github.com/user-attachments/assets/7a4c9a05-7445-4bf8-b2fd-4c6b1ae4ad05
I noticed the CDK deployment had failed as well, looking deeper, I noticed the rollback had left a bunch of buckets behind, and this was causing a failure on the CreateRepository step of the CDK deploy, I manually emptied and deleted the buckets and the deployment was successful.
— Reply to this email directly, view it on GitHub https://github.com/aws-samples/one-observability-demo/issues/295#issuecomment-2574001712, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEDT32WGHZZNE4GPNYQCYDL2JL4G3AVCNFSM6AAAAABSKNKEG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZUGAYDCNZRGI . You are receiving this because you were mentioned.Message ID: @.***>
Hello @psantus Thank you for your patience. It's been a busy start to the year. I'm going to attempt to deploy via CloudShell in EU-WEST-1 In a new account to verify if it builds. I'll let you know what I find.
I had seen the issue that grhartt-aws mentioned previously with an account I had used before. Not sure if that is your issue but let me know.
any result? I haven't yet managed to deploy it myself, and just like you it's been busy.
Thanks for your patience.
Could you try to deploy again?
There have been some updates recently that may have addressed your issue.
If your build fails on WaitCondition for CodePipeline, the CloudWatch Log group for /codebuild/PipelineDeployProject will have the logs for the build and the failure should be listed in those logs.
Failed today on Services stack (loadBalancerCRDEF35F65C) with a different error:
Error in CloudFormation:
Received response status [FAILED] from custom resource. Message returned: TooManyRequestsException: Rate Exceeded.
at de_TooManyRequestsExceptionRes (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:4348:21)
at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:3728:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
at async invokeUserFunction (/var/task/framework.js:1:2794)
at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1837)
(RequestId: 9b934b78-3439-492f-96a2-14ebe95fc234)
Hi team, I encountered the same error as psantus in multiple regions -- US-West-2, US-East-2, and AP-Southeast-1
Hello I think It's a duplicate of the issue https://github.com/aws-samples/one-observability-demo/issues/317 It worked for me with https://github.com/aws-samples/one-observability-demo/pull/394
