amplify-backend icon indicating copy to clipboard operation
amplify-backend copied to clipboard

CDK bootstrap detection assumes stack name of `CDKToolkit`

Open OperationalFallacy opened this issue 1 year ago • 22 comments

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v20.9.0

Amplify CLI Version

12.10.0

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No, this is a brand new project

Describe the bug

npm create amplify@latest completes

npx ampx sandbox --profile xxx-sso-xxx-dev fails with

The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox.

Expected behavior

It must use bootstrapped CDK already in the account.

Reproduction steps

CDK bootstrapped in account

  1. npm create amplify@latest

  2. npx ampx sandbox --profile xxx-sso-xxx-dev

Project Identifier

n/a

Log output

# Put your logs below this line


Additional information

Looking at the web-page the cli opens:

First-time account setup required
Amplify needs to run a one-time set up for this account and region before it can deploy resources. This usually takes between 2 to 5 minutes.
[Learn more](https://docs.amplify.aws/gen2/start/account-setup/)

and checking the link to the docs, I find a long and manual process to setup SSO and bootstrap CDK. I would expect find some information how it should work within existing SSO, and bootstrapped CDK

image

Also, other problems related to getting started. Gen2 UI lists gen1 apps here (https://us-east-1.console.aws.amazon.com/amplify/apps - why it's even listing them???) and offers to delete it 😱

image

Before submitting, please confirm:

  • [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
  • [X] I have removed any sensitive information from my code snippets and submission.

OperationalFallacy avatar May 12 '24 20:05 OperationalFallacy

I'm having the same problem. Trying to create a sandbox environment using npx ampx sandbox gives the output The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox. and opens the AWS Management Console. But there it shows the message "CDKToolkit is already setup successfully in eu-central-1 Region. You may now close this browser window.".

When i run npx ampx sandbox again it does the same thing. So i'm stuck here and can't launch a sandbox environment. I followed the guide here https://docs.amplify.aws/react/start/account-setup/#configure-amplify-cicd.

Both the Amplify app as well as the IAM Identity Center user are in the same region (eu-central-1) and the local profile seems correct too:

tail ~/.aws/config

[default]
region = eu-central-1
sso_session = amplify-admin
sso_account_id = XXXX
sso_role_name = amplify-policy
[sso-session amplify-admin]
sso_start_url = https://XXXX.awsapps.com/start
sso_region = eu-central-1
sso_registration_scopes = sso:account:access

thomasoehri avatar May 14 '24 14:05 thomasoehri

Can you check if you have a CDKToolkit named stack in your region and if it's there, what's the value of BootstrapVersion in the CFN outputs. Reference

Screenshot 2024-05-14 at 17 18 38

Amplifiyer avatar May 14 '24 15:05 Amplifiyer

Hi @thomasoehri, this could also be caused by a region mismatch. Is the AWS_REGION or AWS_DEFAULT_REGION environment variable set? If so, this will override the region in the config file.

edwardfoyle avatar May 14 '24 15:05 edwardfoyle

Can you check if you have a CDKToolkit named stack in your region and if it's there, what's the value of BootstrapVersion in the CFN outputs. Reference

Screenshot 2024-05-14 at 17 18 38

Yes, the CDKToolkit named stack is present in region eu-central-1 and showing a BootstrapVersion value of 18:

Bildschirmfoto 2024-05-14 um 18 01 54

thomasoehri avatar May 14 '24 16:05 thomasoehri

in mine case, the version 14

OperationalFallacy avatar May 14 '24 16:05 OperationalFallacy

Hi @thomasoehri, this could also be caused by a region mismatch. Is the AWS_REGION or AWS_DEFAULT_REGION environment variable set? If so, this will override the region in the config file.

Hi @edwardfoyle, thank you for the great tip, but i have neither of them set.

thomasoehri avatar May 14 '24 17:05 thomasoehri

to debug this next, can you run the following commands and let us know the results

  1. aws ec2 describe-availability-zones --output text --query 'AvailabilityZones[0].[RegionName]' --profile xxx-sso-xxx-dev | cat

  2. aws cloudformation describe-stacks --stack-name CDKToolkit --max-items 1 --profile xxx-sso-xxx-dev | grep BootstrapVersion -B 1 -A 3

I'm assuming you are still getting this error:

The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbo

Amplifiyer avatar May 16 '24 11:05 Amplifiyer

to debug this next, can you run the following commands and let us know the results

  1. aws ec2 describe-availability-zones --output text --query 'AvailabilityZones[0].[RegionName]' --profile xxx-sso-xxx-dev | cat
  2. aws cloudformation describe-stacks --stack-name CDKToolkit --max-items 1 --profile xxx-sso-xxx-dev | grep BootstrapVersion -B 1 -A 3

I'm assuming you are still getting this error:

The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbo
  1. While running the first command i received the following authorization error: An error occurred (UnauthorizedOperation) when calling the DescribeAvailabilityZones operation: You are not authorized to perform this operation. User: arn:aws:sts::xxxxx:assumed-role/AWSReservedSSO_amplify-policy_xxxxx/amplify-admin is not authorized to perform: ec2:DescribeAvailabilityZones because no identity-based policy allows the ec2:DescribeAvailabilityZones action

I then attached the AmazonEC2ReadOnlyAccess policy to the IAM Identity Center permission set for my user and got the following response when running the command again:

thomasoehri@Mac-Studio-von-Thomas-572 echo-backend % aws ec2 describe-availability-zones --output text --query 'AvailabilityZones[0].[RegionName]' --profile default | cat
eu-central-1
  1. Running the second command i got the following response:
thomasoehri@Mac-Studio-von-Thomas-572 echo-backend % aws cloudformation describe-stacks --stack-name CDKToolkit --max-items 1 --profile default | grep BootstrapVersion -B 1 -A 3
                {
                    "OutputKey": "BootstrapVersion",
                    "OutputValue": "20",
                    "Description": "The version of the bootstrap resources that are currently mastered in this stack"
                },

P.S.: It is now showing BootstrapVersion 20 instead of 18 like before because i tried updating the CDKToolkit stack manually using the command cdk bootstrap aws://$(aws sts get-caller-identity --query Account --output text)/$AWS_REGION from the following guide: Troubleshoot "Stack CDKToolkit already exists", but that didn't fix the error The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox..

  1. Running npx ampx sandbox --profile default still results in the The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox. error.

Here is a screenshot showing all three commands and their results:

Bildschirmfoto 2024-05-16 um 14 22 23 Bildschirmfoto 2024-05-16 um 14 22 02

Note that i use the same profile --profile default with every command.

thomasoehri avatar May 16 '24 12:05 thomasoehri

Interesting! This is the code that makes the same CFN call to determine if the account-region is bootstrapped.

Would it be possible for you to put a breakpoint in that method and see what you get back? Also if you are using a default profile, what do you see if you don't use the --profile option at all.

Amplifiyer avatar May 16 '24 13:05 Amplifiyer

Interesting! This is the code that makes the same CFN call to determine if the account-region is bootstrapped.

Would it be possible for you to put a breakpoint in that method and see what you get back? Also if you are using a default profile, what do you see if you don't use the --profile option at all.

Here is what i get back when i set a breakpoint in the isBootstrapped method:

Bildschirmfoto 2024-05-16 um 17 26 04

This call throws an error with code 'ValidationError' and message 'Stack with id CDKToolkit does not exist':

const { Stacks: stacks } = await this.cfnClient.send(
        new DescribeStacksCommand({
          StackName: CDK_BOOTSTRAP_STACK_NAME,
        })
      );

The --profile argument was just to showcase that the correct/same profile was used. Omitting it also results in the error.

thomasoehri avatar May 16 '24 15:05 thomasoehri

this.cfnClient.config.region() seems correct too:

Bildschirmfoto 2024-05-16 um 17 58 53

thomasoehri avatar May 16 '24 16:05 thomasoehri

I have found the issue (at least for me). I've had credentials for another AWS account stored in ~/.aws/credentials and it seems ampx sandbox took those instead of the SSO profile stored in ~/.aws/config whereas all other cli commands took the SSO profile. Maybe that is something to add to the troubleshooting page.

thomasoehri avatar May 16 '24 16:05 thomasoehri

Thank you @thomasoehri so much for you help in debugging this.

I have found the issue (at least for me). I've had credentials for another AWS account stored in ~/.aws/credentials and it seems ampx sandbox took those instead of the SSO profile stored in ~/.aws/config whereas all other cli commands took the SSO profile.

That was my next guess that aws-sdk is not able to resolve the right AWS_ACCOUNT since everything else lines up. See some issues filed there

Maybe that is something to add to the troubleshooting page.

Thank you, yes we will look into adding this there. Can you share the snippet of your Obfuscated credentials file on how it looked? I'm guessing if the same profile name is present there, aws-sdk picks that ahead of the aws/config

Amplifiyer avatar May 16 '24 16:05 Amplifiyer

@OperationalFallacy, can you check if your situation is similar to the one we just discussed above?

Amplifiyer avatar May 16 '24 16:05 Amplifiyer

@OperationalFallacy, can you check if your situation is similar to the one we just discussed above?

I see it recognizes sso profile (both with a workaround for gen1 - credential_process = aws-sso-credential-process --profile xxx and normal sso profile), the account is correct, too.

npx ampx sandbox --profile xxx-dev-amplify
The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox.

There is still the prompt.

image

Looks like a bug because the latest cdk cli works just fine, even with some older bootstrap version I have.

OperationalFallacy avatar May 16 '24 16:05 OperationalFallacy

what do you get with this command aws cloudformation describe-stacks --stack-name CDKToolkit --max-items 1 --profile xxx-sso-xxx-dev | grep BootstrapVersion -B 1 -A 3

Amplifiyer avatar May 16 '24 17:05 Amplifiyer

aws cloudformation describe-stacks --stack-name CDKToolkit-CICDTarget --max-items 1 --xxx-dev-amplify | grep BootstrapVersion -B 1 -A 3
                {
                    "OutputKey": "BootstrapVersion",
                    "OutputValue": "14",
                    "Description": "The version of the bootstrap resources that are currently mastered in this stack"
                },
             ```
             
 Note the custom stack name.
 
 I guess it should be more like this or whatever the sdk call to get the output value by output name 

aws cloudformation describe-stacks --query "Stacks[].Outputs[?OutputKey=='BootstrapVersion'].[OutputValue]" --output text --profile xxxx``` - it prints 14

OperationalFallacy avatar May 16 '24 17:05 OperationalFallacy

Thanks @OperationalFallacy, this is helpful. The code currently assumes that the stack-name has to be CDKToolkit which is the default when using cdk bootstrap. Marking this a bug.

Amplifiyer avatar May 16 '24 17:05 Amplifiyer

Thank you, yes we will look into adding this there. Can you share the snippet of your Obfuscated credentials file on how it looked? I'm guessing if the same profile name is present there, aws-sdk picks that ahead of the aws/config

Yes exactly, both profiles were named default.

% cat ~/.aws/credentials
[default]
aws_access_key_id=XXXXX
aws_secret_access_key=XXXXX

thomasoehri avatar May 17 '24 09:05 thomasoehri

Just ran into this myself. I'm a little bummed because I wanted to participate in the AWS Amplify full stack challenge

credentials

[default]
aws_access_key_id = xxx
aws_secret_access_key = xxx

config

[default]
sso_session = amplify-admin
sso_account_id = xxx
sso_role_name = amplify-policy
region = us-east-2
[sso-session amplify-admin]
sso_start_url = https://xxx.awsapps.com/start
sso_region = us-east-2
sso_registration_scopes = sso:account:access

robokozo avatar May 19 '24 04:05 robokozo

@robokozo You should be able to give the SSO profile a different name from the profile in ~/.aws/credentials. So when running aws configure sso choose a different profile name:

% aws configure sso
SSO session name [amplify-admin]:
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.eu-central-1.amazonaws.com/

Then enter the code:

XXXX-XXXX
The only AWS account available to you is: XXXXX
Using the account ID XXXXX
The only role available to you is: amplify-policy
Using the role name "amplify-policy"
CLI default client Region [eu-central-1]:
CLI default output format [None]:
CLI profile name [amplify-policy-533267102941]: amplify-policy-XXX  <---- Use a profile name other than default

To use this profile, specify the profile name using --profile, as shown:

aws s3 ls --profile amplify-policy-XXX

Then you'll be able to run the sandbox with that profile using npx ampx sandbox --profile amplify-policy-XXX. In my case i've just removed the ~/.aws/credentials file since i didn't need the account that was specified in there on this machine.

thomasoehri avatar May 19 '24 08:05 thomasoehri

@thomasoehri Thanks for the info. Any idea about this?

> npx ampx sandbox --profile amplify-policy-my-name

[Sandbox] Pattern !.env.example found in .gitignore. ".env.example" will not be watched if other patterns in .gitignore are excluding it.
  
  Amplify Sandbox
  
  Identifier:   my-name
  Stack:        amplify-my-name-sandbox-eb92868d9f
  
  To specify a different sandbox identifier, use --identifier
Segmentation fault

robokozo avatar May 19 '24 15:05 robokozo

npx ampx sandbox is working now, thank you for the update!

Is the UI trying to do something different when it creates an app? A few Gen 1 apps are already configured and deployed in that account, so the message is confusing.

image

What is it trying to do?

OperationalFallacy avatar Jun 19 '24 14:06 OperationalFallacy

I just encounter this issue as well but I was able to get it to work once I've removed ~/aws/.credentials

jakejcheng avatar Jul 26 '24 02:07 jakejcheng

I also got the same issue with same message The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox. when running npx ampx sandbox command without --profile profile-name option, and when added the profile option, I was getting error related to invalidURL and found the issue in my aws credentials. AWS region value was setup wrong, correcting it fixed the issue for me.

harnam2601 avatar Sep 10 '24 12:09 harnam2601

When running this command 'npx ampx sandbox --outputs-format dart --outputs-out-dir lib' getting this error 'The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox.'

Run this command to fix it: 'npx aws-cdk@latest bootstrap aws://540444658130/us-east-1'

zeeshanakhter2009 avatar Oct 10 '24 16:10 zeeshanakhter2009

this last comment from @zeeshanakhter2009 worked for me as well structure of the cli command is as follows

npx aws-cdk@latest bootstrap aws://{AWS account Number}/{region}

e.g. npx aws-cdk@latest bootstrap aws://123456789012/us-east-1

cpropster avatar Oct 11 '24 19:10 cpropster

When running this command 'npx ampx sandbox --outputs-format dart --outputs-out-dir lib' getting this error 'The given region has not been bootstrapped. Sign in to console as a Root user or Admin to complete the bootstrap process, then restart the sandbox.'

Run this command to fix it: 'npx aws-cdk@latest bootstrap aws://540444658130/us-east-1'

This finally worked for me, with one caveat: the amplify-policy role created if you use the process in the guide linked above uses AmplifyBackendDeployFullAccess policy, which doesn't have CloudFormation:CreateChangeSet permission, which is required for the bootstrap. You'll need to add that to the role or just use an AdministratorAccess role. In ~/.aws/config change your sso profile sso_role_name setting to use the role you need.

tal-arrowood avatar Oct 16 '24 20:10 tal-arrowood