amazon-genomics-cli
amazon-genomics-cli copied to clipboard
Published
20 hours ago •
aws
aws
AGC not working with enforced S3 encryption
Describe the Bug The only way how AWS allows to enforce encryption on S3 is using SCP: https://aws.amazon.com/blogs/security/how-to-prevent-uploads-of-unencrypted-objects-to-amazon-s3/
AGC does not work when S3 encryption is being enforced through SCP. Most larger enterprises use such a mechanism to avoid upload of unencrypted data.
Steps to Reproduce
- Setup an account A with AWS Organizations and the below SCP
- Add another account B to this organization
- Make sure to attach the below policy so that it applies to Account B.
-
agc account activate-> will fail due to S3 encryption header enforced
{
"Version": "2012-10-17",
"Statement": [
{
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": [
"AES256",
"aws:kms"
]
},
"ArnNotLike": {
"aws:PrincipalARN": [
"arn:aws:iam::*:role/XXX_*"
]
}
},
"Action": [
"s3:PutObject"
],
"Resource": "*",
"Effect": "Deny",
"Sid": "DenyUnencryptedObjectUploads"
},
{
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
},
"ArnNotLike": {
"aws:PrincipalARN": [
"arn:aws:iam::*:role/abc_logs",
"arn:aws:iam::*:role/XXXX_*"
]
}
},
"Action": "s3:PutObject",
"Resource": "*",
"Effect": "Deny",
"Sid": "DenyIncorrectEncryptionHeader"
}
]
}
Relevant Logs 11 //: # (The logs that where obtained by running the commands here. Please run the command with "-v" so that we can see the verbose logs.)
Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
Admin:~/environment $ agc account activate --vpc vpc-051647cf231f041b8 --subnets subnet-06f8288cdb3201075 --subnets subnet-0d060254c7bd29f95
2022-11-23T07:56:42Z 𝒊 Activating AGC with bucket '' and VPC 'vpc-051647cf231f041b8'
Bootstrapping CDK... [-o--] 27s
Activating account... [-o--] 2m34s 2022-11-23T07:59:43Z ✘ [WARNING] aws-cdk-lib.aws_ssm.StringParameterProps#type is deprecated.
2022-11-23T07:59:43Z ✘ - type will always be 'String'
2022-11-23T07:59:43Z ✘ This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘ [WARNING] aws-cdk-lib.aws_ssm.ParameterType is deprecated.
2022-11-23T07:59:43Z ✘ these types are no longer used
2022-11-23T07:59:43Z ✘ This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘ [WARNING] aws-cdk-lib.aws_ssm.ParameterType#STRING is deprecated.
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘ [Warning at /Agc-Core/InfraSubnet0] No routeTableId was provided to the subnet 'subnet-06f8288cdb3201075'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
2022-11-23T07:59:43Z ✘ [Warning at /Agc-Core/InfraSubnet1] No routeTableId was provided to the subnet 'subnet-0d060254c7bd29f95'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ ✨ Synthesis time: 10.4s
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ Agc-Core: building assets...
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-deploy-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ [0%] start: Building c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Building 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Building 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Building 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Building 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [20%] success: Built c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [40%] success: Built 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [60%] success: Built 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [80%] success: Built 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [100%] success: Built 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ Agc-Core: assets built
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ Agc-Core: deploying...
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-deploy-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ [0%] start: Publishing c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Publishing 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Publishing 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Publishing 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [0%] start: Publishing 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘ [20%] success: Published 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [40%] success: Published 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [60%] success: Published c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [80%] success: Published 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ [100%] success: Published 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘ Agc-Core: creating CloudFormation changeset...
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:23 AM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | Agc-Core User Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:28 AM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | Agc-Core User Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:33 AM | CREATE_IN_PROGRESS | AWS::DynamoDB::Table | Table (TableCD117FA1)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:33 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | vpc (vpcA2121C38)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:33 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:33 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | bucket (bucket43879C71)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | ComputeEnvImage (ComputeEnvImage84B45428)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | NumInfraSubnets (NumInfraSubnets35FDF285)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::IAM::Role | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::Lambda::LayerVersion | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | InfraSubnets (InfraSubnets06E8F9B3)
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:34 AM | CREATE_IN_PROGRESS | AWS::IAM::Role | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:35 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:35 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | bucket (bucket43879C71) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:35 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | vpc (vpcA2121C38) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:35 AM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:36 AM | CREATE_IN_PROGRESS | AWS::DynamoDB::Table | Table (TableCD117FA1) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 0/15 | 7:57:36 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | ComputeEnvImage (ComputeEnvImage84B45428) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 1/15 | 7:57:36 AM | CREATE_COMPLETE | AWS::SSM::Parameter | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53)
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:57:36 AM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:57:36 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:57:36 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | NumInfraSubnets (NumInfraSubnets35FDF285) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:57:36 AM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | InfraSubnets (InfraSubnets06E8F9B3) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 3/15 | 7:57:36 AM | CREATE_COMPLETE | AWS::SSM::Parameter | bucket (bucket43879C71)
2022-11-23T07:59:43Z ✘ Agc-Core | 4/15 | 7:57:36 AM | CREATE_COMPLETE | AWS::SSM::Parameter | vpc (vpcA2121C38)
2022-11-23T07:59:43Z ✘ Agc-Core | 5/15 | 7:57:37 AM | CREATE_COMPLETE | AWS::SSM::Parameter | ComputeEnvImage (ComputeEnvImage84B45428)
2022-11-23T07:59:43Z ✘ Agc-Core | 6/15 | 7:57:37 AM | CREATE_COMPLETE | AWS::SSM::Parameter | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F)
2022-11-23T07:59:43Z ✘ Agc-Core | 7/15 | 7:57:37 AM | CREATE_COMPLETE | AWS::SSM::Parameter | InfraSubnets (InfraSubnets06E8F9B3)
2022-11-23T07:59:43Z ✘ Agc-Core | 8/15 | 7:57:37 AM | CREATE_COMPLETE | AWS::SSM::Parameter | NumInfraSubnets (NumInfraSubnets35FDF285)
2022-11-23T07:59:43Z ✘ Agc-Core | 8/15 | 7:57:43 AM | CREATE_IN_PROGRESS | AWS::Lambda::LayerVersion | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 9/15 | 7:57:43 AM | CREATE_COMPLETE | AWS::Lambda::LayerVersion | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C)
2022-11-23T07:59:43Z ✘ Agc-Core | 10/15 | 7:57:52 AM | CREATE_COMPLETE | AWS::IAM::Role | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265)
2022-11-23T07:59:43Z ✘ Agc-Core | 10/15 | 7:57:54 AM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF)
2022-11-23T07:59:43Z ✘ Agc-Core | 10/15 | 7:57:55 AM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 11/15 | 7:58:00 AM | CREATE_COMPLETE | AWS::DynamoDB::Table | Table (TableCD117FA1)
2022-11-23T07:59:43Z ✘ Agc-Core | 12/15 | 7:58:13 AM | CREATE_COMPLETE | AWS::IAM::Policy | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF)
2022-11-23T07:59:43Z ✘ Agc-Core | 12/15 | 7:58:14 AM | CREATE_IN_PROGRESS | AWS::Lambda::Function | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536)
2022-11-23T07:59:43Z ✘ Agc-Core | 12/15 | 7:58:20 AM | CREATE_IN_PROGRESS | AWS::Lambda::Function | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:58:26 AM | CREATE_COMPLETE | AWS::Lambda::Function | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:58:28 AM | CREATE_IN_PROGRESS | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A)
2022-11-23T07:59:43Z ✘ 13/15 Currently in progress: Agc-Core, BatchArtifactsCustomResourceAA86556A
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:09 AM | CREATE_IN_PROGRESS | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Resource creation Initiated
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:09 AM | CREATE_FAILED | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:10 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | Agc-Core The following resource(s) failed to create: [BatchArtifactsCustomResourceAA86556A]. Rollback requested by user.
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::DynamoDB::Table | Table (TableCD117FA1)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | bucket (bucket43879C71)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | NumInfraSubnets (NumInfraSubnets35FDF285)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | vpc (vpcA2121C38)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | ComputeEnvImage (ComputeEnvImage84B45428)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::SSM::Parameter | InfraSubnets (InfraSubnets06E8F9B3)
2022-11-23T07:59:43Z ✘ Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
2022-11-23T07:59:43Z ✘ Agc-Core | 12/15 | 7:59:23 AM | DELETE_COMPLETE | AWS::SSM::Parameter | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53)
2022-11-23T07:59:43Z ✘ Agc-Core | 11/15 | 7:59:23 AM | DELETE_COMPLETE | AWS::SSM::Parameter | bucket (bucket43879C71)
2022-11-23T07:59:43Z ✘ Agc-Core | 10/15 | 7:59:23 AM | DELETE_COMPLETE | AWS::SSM::Parameter | vpc (vpcA2121C38)
2022-11-23T07:59:43Z ✘ Agc-Core | 9/15 | 7:59:24 AM | DELETE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
2022-11-23T07:59:43Z ✘ Agc-Core | 8/15 | 7:59:24 AM | DELETE_COMPLETE | AWS::SSM::Parameter | NumInfraSubnets (NumInfraSubnets35FDF285)
2022-11-23T07:59:43Z ✘ Agc-Core | 7/15 | 7:59:24 AM | DELETE_COMPLETE | AWS::SSM::Parameter | ComputeEnvImage (ComputeEnvImage84B45428)
2022-11-23T07:59:43Z ✘ Agc-Core | 6/15 | 7:59:24 AM | DELETE_COMPLETE | AWS::SSM::Parameter | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F)
2022-11-23T07:59:43Z ✘ Agc-Core | 5/15 | 7:59:24 AM | DELETE_COMPLETE | AWS::SSM::Parameter | InfraSubnets (InfraSubnets06E8F9B3)
2022-11-23T07:59:43Z ✘ Agc-Core | 6/15 | 7:59:24 AM | DELETE_COMPLETE | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A)
2022-11-23T07:59:43Z ✘ Agc-Core | 6/15 | 7:59:26 AM | DELETE_IN_PROGRESS | AWS::Lambda::Function | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536)
2022-11-23T07:59:43Z ✘ Agc-Core | 5/15 | 7:59:33 AM | DELETE_COMPLETE | AWS::Lambda::Function | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536)
2022-11-23T07:59:43Z ✘ Agc-Core | 5/15 | 7:59:33 AM | DELETE_IN_PROGRESS | AWS::IAM::Policy | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF)
2022-11-23T07:59:43Z ✘ Agc-Core | 5/15 | 7:59:33 AM | DELETE_IN_PROGRESS | AWS::Lambda::LayerVersion | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C)
2022-11-23T07:59:43Z ✘ Agc-Core | 4/15 | 7:59:34 AM | DELETE_COMPLETE | AWS::IAM::Policy | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF)
2022-11-23T07:59:43Z ✘ Agc-Core | 3/15 | 7:59:34 AM | DELETE_COMPLETE | AWS::DynamoDB::Table | Table (TableCD117FA1)
2022-11-23T07:59:43Z ✘ Agc-Core | 3/15 | 7:59:35 AM | DELETE_IN_PROGRESS | AWS::IAM::Role | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265)
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:59:35 AM | DELETE_COMPLETE | AWS::Lambda::LayerVersion | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C)
2022-11-23T07:59:43Z ✘ Agc-Core | 1/15 | 7:59:36 AM | DELETE_COMPLETE | AWS::IAM::Role | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265)
2022-11-23T07:59:43Z ✘ Agc-Core | 2/15 | 7:59:36 AM | ROLLBACK_COMPLETE | AWS::CloudFormation::Stack | Agc-Core
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ Failed resources:
2022-11-23T07:59:43Z ✘ Agc-Core | 7:59:09 AM | CREATE_FAILED | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ ❌ Agc-Core failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘ at FullCloudFormationDeployment.monitorDeployment (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/api/deploy-stack.ts:496:13)
2022-11-23T07:59:43Z ✘ at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-23T07:59:43Z ✘ at deployStack2 (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:241:24)
2022-11-23T07:59:43Z ✘ at /home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/deploy.ts:39:11
2022-11-23T07:59:43Z ✘ at run (/home/ec2-user/.agc/cdk/node_modules/p-queue/dist/index.js:163:29)
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ ❌ Deployment failed: Error: Stack Deployments Failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘ at deployStacks (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/deploy.ts:61:11)
2022-11-23T07:59:43Z ✘ at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-23T07:59:43Z ✘ at CdkToolkit.deploy (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:314:7)
2022-11-23T07:59:43Z ✘ at initCommandLine (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cli.ts:357:12)
2022-11-23T07:59:43Z ✘
2022-11-23T07:59:43Z ✘ Stack Deployments Failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘ error="exit status 1"
Error: an error occurred invoking 'account activate'
with variables: {bucketName: vpcId:vpc-051647cf231f041b8 publicSubnets:false customTags:map[] subnets:[subnet-06f8288cdb3201075 subnet-0d060254c7bd29f95] amiId:}
caused by: exit status 1
Expected Behavior
The following modifcation of the AGC code core-stack.ts resolves this issue by sending along the encryption header when uploading:
new BucketDeployment(this, "BatchArtifacts", {
sources: [Source.asset(path.join(__dirname, "../artifacts"))],
destinationBucket: this.bucket,
destinationKeyPrefix: "artifacts",
prune: false,
metadata: {
"idempotency-key": props.idempotencyKey,
},
serverSideEncryption: ServerSideEncryption.AES_256,
});
Actual Behavior
Screenshots
Additional Context
Suggested Implementation: see the TODOs in the following branch https://github.com/aws/amazon-genomics-cli/compare/main...ArlindNocaj:amazon-genomics-cli:feature/sse-headers
Operating System: AGC Version: Was AGC setup with a custom bucket: Was AGC setup with a custom VPC:
