aws
(amplify-alpha): CDK wants to zip and upload whole project folder
Describe the bug
A cdk deploy run tries to zip and upload the whole project folder including .git, node_modules, etc.
Expected Behavior
Only the Asset itself is zipped and uploaded, not the whole project folder.
Current Behavior
I'm working on a CDK monorepo project, so the frontend is only a part of the project.
The structure of the project looks like this: packages\module
Where module is a Lambda, API Gateway, whatever and the Vue.js frontend.
So the frontend is located in packages\vue-frontend.
Currently, the frontend is build locally and is available in packages\vue-frontend\dist for the cdk deploy run. The current version uses a BucketDeployment to upload the files and the frontend is published using CloudFront. This works fine so far.
Now there is the requirement to use Amplify to host the frontend. So the idea is to use amplify-alpha to use an Asset to upload the static frontend build. Unfortunately, using GitHub, GitLab or CodeCommit isn't an option so I can't rely on a SourceCodeProvider.
[16:35:46] my-project: build: Zip C:\Users\Aldjinn\Projects\my-project -> C:\Users\wilm028\Projects\my-project\cdk.out.cache\acbcce3d7c7707096b4a0d04c139c1f2d642d3757294ce0d1ba609b00a31d91a.zip
This file ends up with > 2 GB, if the zip even finishes.
Reproduction Steps
const amplifyApp = new App(this, 'MyApp', {
appName: "my-app",
});
const asset = new assets.Asset(this, 'SampleAsset', {
path: path.join(__dirname, 'dist'),
});
const branch = amplifyApp.addBranch('dev', {
asset: asset,
});
The problem is only triggered when Branch and Asset are added. Without these, everything works fine.
Possible Solution
No response
Additional Information/Context
The required zip for the Asset is created. But I don't understand, which part of my project triggers this big zip of the whole project folder.
CDK CLI Version
2.128.0 (build d995261)
Framework Version
2.128.0-alpha.0
Node.js Version
v18.19.1
OS
Windows 11 22H2
Language
TypeScript
Language Version
5.2.2
Other information
No response
It's unclear to me what your directories look like.
If you define your stack in the lib/demo.ts like this:
export class DummyStack extends Stack {
constructor(scope: Construct, id: string, props: StackProps) {
super(scope, id, props);
const amplifyApp = new amplify_alpha.App(this, 'MyApp', {
appName: 'my-app'
});
const asset = new s3assets.Asset(this, 'SampleAsset', {
path: path.join(__dirname, 'dist'),
});
const branch = amplifyApp.addBranch('dev', {
asset: asset,
});
}
}
You are supposed to have lib/dist for the Asset and it should not bundle up the cdk.out directory which causes the error.
I've tried several approaches with different locations of the "dist" folder, like...
Inside the module: C:\Users\Aldjinn\Projects\my-project\packages\module\dist
In the root folder of the project: C:\Users\Aldjinn\Projects\my-project\dist
Even outside of the project folder: C:\Temp\dist
Always with the same result. I'm unfortunately not able to share the whole project. But I tried to disable everything but the Amplify stuff and the problem still remains. And an empty "hello world project" with only Amplify works.
In addition, I was able to find out that the asset for the Amplify upload is generated correctly, i.e. a ZIP with the content from the "dist" directory. But the problem of the ZIP of the whole project remains. And I don't why this is triggered.
I was able to narrow the problem down: It only occurs when I'm using the "-e" or "--exclusively" flag to deploy only changes in the current stack and not on other (shared) stacks within the project.
# creates the huge unwanted zip file
cdk deploy -e my-stack
# works fine
cdk deploy
This can be reproduced with cdk synth as well.
That looks like the same problem, but quite old: https://github.com/aws/aws-cdk/issues/12898
I am able to reproduce this issue consistently as well when using the -e or --exclusively option as well in my project with following code snippet. The issue is not encountered when simply using cdk synth or cdk deploy without the -e.
const asset = new assets.Asset(this, 'SampleAsset', {
path: path.join(__dirname, 'sample-asset-directory'),
});
const amplifyApp = new amplify.App(this, 'MyApp', {});
const branch = amplifyApp.addBranch("dev", { asset: asset });
My Project (AmplifyTest) directory structure is as follows -
.
├── bin
│ ├── amplify_test.d.ts
│ ├── amplify_test.js
│ └── amplify_test.ts
├── cdk.json
├── jest.config.js
├── lib
│ ├── amplify_test-stack.d.ts
│ ├── amplify_test-stack.js
│ ├── amplify_test-stack.ts
│ └── sample-asset-directory
│ └── my_name.txt
The AmplifyTestStack.assets.json file contains the following extra asset which zips the entire project directory in addition to the correct Asset limited to the sample-asset-directory -
"0dc14b69c0e1671607cab7f6c2f50133555ebcc89d5d61e3801f92d47b8eaaf1": {
"source": {
"path": "/home/ec2-user/environment/AmplifyTest",
"packaging": "zip"
},
"destinations": {
"current_account-current_region": {
"bucketName": "cdk-123456789-assets-${AWS::AccountId}-${AWS::Region}",
"objectKey": "0dc14b69c0e1671607cab7f6c2f50133555ebcc89d5d61e3801f92d47b8eaaf1.zip",
"assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-123456789-file-publishing-role-${AWS::AccountId}-${AWS::Region}"
}
}
},
~/environment/AmplifyTest (master) $ cdk --version
2.128.0 (build d995261)
Perhaps the approach from the CDK pipeline reference architecture could help here?
Basically, as you can see in the source here, they use .gitignore to exclude files from the Asset:
...
let gitignore = fs.readFileSync('.gitignore').toString().split(/\r?\n/);
gitignore.push('.git/');
...
gitignore = gitignore.filter(g => g != 'node_modules/');
gitignore.push('/node_modules/');
const codeAsset = new Asset(this, 'SourceAsset', {
path: '.',
ignoreMode: IgnoreMode.GIT,
exclude: gitignore,
});
...
