copilot-cli icon indicating copy to clipboard operation
copilot-cli copied to clipboard

[Bug]: Can't deploy because of Secrets issue

Open apassy opened this issue 10 months ago • 8 comments

Other Closed Issues related to tagging.

Description:

I'm trying to deploy a service that I previously deployed just fine, but I added a secrets section to the manifest, and now it's failing.

Details:

Copilot ver: 1.33.1 running on Windows 11 Load-balanced web app

Additional manifest lines:

secrets:                      
  adc_reader:
    secretsmanager: 'ReportWriter_ADC_DB'

Error:

  - [a7438eb0]: ResourceInitializationError: unable to pull secrets or reg
    istry auth: execution resource retrieval failed: unable to retrieve se
    cret from asm: service call has been retried 1 time(s): failed to fetc
    h secret arn:aws:secretsmanager:us-east-1:<ID REDACTED>:secret:ReportWr
    iter_ADC_DB from secrets manager: AccessDeniedException: User: arn:aws
    :sts::<ID REDACTED>:assumed-role/streamlit-sample-test-front-end-Execut
    ionRole-qXmoAXceH13T/a7438eb04296469cbb925934135fa489 is not authorize
    d to perform: secretsmanager:GetSecretValue on resource: arn:aws:secre
    tsmanager:us-east-1:<ID REDACTED>:secret:ReportWriter_ADC_DB because no
     identity-based policy allows the secretsmanager:GetSecretValue action
     status code: 400, request id: 53f8285a-ebbf-4208-8358-c011a9c0a9fc

Secrets are tagged with the copilot-application and copilot-environment and those match what I'm using.

Expected result:

expected successful deployment

apassy avatar Apr 03 '24 20:04 apassy

This has been reported in #5732. You need to give the full ARN of the secret, not just its name.

al-dpopowich avatar Apr 04 '24 13:04 al-dpopowich

Same error when using the full ARN.

apassy avatar Apr 04 '24 23:04 apassy

hey @apassy , please see my response here and see if it help clarify anything for you. In the meantime, can you try specifying the secret arn instead of just the name?

Lou1415926 avatar Apr 05 '24 20:04 Lou1415926

Tried with full ARN

secrets:                      # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
  adc_reader:
    secretsmanager: 'arn:aws:secretsmanager:us-east-1:<acct>:secret:ReportWriter_ADC_DB-<random>'
  dropbox_writer:
    secretsmanager: 'arn:aws:secretsmanager:us-east-1:<acct>:secret:DropboxReportWriter-<random>'
  infra_reader:
    secretsmanager: 'arn:aws:secretsmanager:us-east-1:<acct>:secret:ReportWriter_Infrastructure_DB-<random>'

    ✘ Latest 2 tasks stopped reason
      - [955086cd,9b359892]: ResourceInitializationError: unable to pull secre
        ts or registry auth: execution resource retrieval failed: unable to re
        trieve secret from asm: service call has been retried 1 time(s): secre
        ts manager: failed to retrieve secret from arn:aws:secretsmanager:us-e
        ast-1:<acct>:secret:arn:aws:secretsmanager:us-east-1:49115046704
        7:secret:ReportWriter_ADC_DB-<random>: unexpected ARN format with parame
        ters when trying to retrieve ASM secret

apassy avatar Apr 08 '24 14:04 apassy

@apassy remove the _ and any - in the last segment of your secret.

@Lou1415926 we just had this issue where our copilot services could not access secrets where the last segment had a hyphen. Example that didn't work: common/data/lookup-id Example that did work common/data/lookupid

It appears if the hyphen is in a previous segment, but not the ending segment, it's fine such as: Works fine: api-common/lookupid

No matter what you'll get an error about accessing the secret.

h5aaimtron avatar Apr 12 '24 00:04 h5aaimtron

I think this is still a bug, all my secrets are working fine in my first environment but now I'm seeing this error when trying to deploy to a new environment

ssyberg avatar May 08 '24 17:05 ssyberg

@ssyberg can you check if the secrets are properly tagged with copilot-application etc.? Are you using the same secrets for both envs?

iamhopaul123 avatar May 10 '24 17:05 iamhopaul123

@ssyberg can you check if the secrets are properly tagged with copilot-application etc.? Are you using the same secrets for both envs?

It was totally the tagging, I missed that sentence in the docs!

ssyberg avatar May 10 '24 18:05 ssyberg

This issue is stale because it has been open 60 days with no response activity. Remove the stale label, add a comment, or this will be closed in 14 days.

github-actions[bot] avatar Jul 10 '24 00:07 github-actions[bot]

This issue is closed due to inactivity. Feel free to reopen the issue if you have any further questions!

github-actions[bot] avatar Jul 24 '24 00:07 github-actions[bot]