Better guidance for using `amazon/aws-codebuild-local`

[jbaileyashe]

I've been trying to use amazon/aws-codebuild-local to debug some build failures, and it's not clear at all how the environment variables are used. Some guidance and examples would be really welcome. In particular:

• IMAGE - Took a long time for me to realize I had to build my own local version of the codebuild images in order to specify a value here. I thought I could use the codebuild images (aws/codebuild/standard:2.0) but obviously doesn't work.
• SOURCE - Does codebuild expect to download my source from GitHub or similar, or can I just mount it into the container?
• aws credentials - Does the local container need my credentials for anything? How do I pass them along?

Finally, I'm on Windows and the fact that codebuild_build.sh is a bash script is painful. I've had to reverse engineer it to figure out how it is calling the amazon/aws-codebuild-local image.

[wayne-folkes]

Using this should be simpler. I just want to be able to test as much of my build before having it run in the code build environment.

[holyjak]

I struggle with nr. 3, passing AWS credentials to the container. I run it in a shell where I have AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN yet running aws CLI during the build fails:

[Container] 2020/01/24 10:30:11 Running command $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION)
agent_1  | Unable to locate credentials. You can configure credentials by running "aws configure".

[kjamaal]

I struggle with nr. 3, passing AWS credentials to the container. I run it in a shell where I have AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN yet running aws CLI during the build fails:

[Container] 2020/01/24 10:30:11 Running command $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION)
agent_1  | Unable to locate credentials. You can configure credentials by running "aws configure".

You should be able to use the -c switch. I'm using my local ~/.aws/credentials and AWS_PROFILE and that works great like ./codebuild_build.sh -i [image] -a result -c.

Where I'm struggling is the source, specifically so I can use and test environment variables that depend on it, like CODEBUILD_RESOLVED_SOURCE_VERSION.

edit: for my env vars issue I'm working around it using -e and mocking the expected value ./codebuild_build.sh -i [image] -a result -c -e .env

[cnelson]

An item I think is important from #312 which was closed to consolidate into this issue is this:

env > secrets-manager > key requires ARN instead of also supporting name (as in-cloud CodeBuild)

We have buildspec files that get built in multiple accounts / regions and we rely on this feature to have the buildspec file grab the right secret based on where it's running so we can't commit full arns to our buildspecs.

Ideally the local agent would respect the AWS_REGION / AWS_DEFAULT_REGION and current caller account id when attempting to grab secrets by name only.

Thanks for all the work on this ❤️

[pandithdurai]

I am using amazon/aws-codebuild-local with codebuild_build.sh script but I am getting Runtime error (*clienterr.PhaseContextError: YAML file does not exist) even I have buildspec.yml in the source directory

 25/08/2020   14:11.17   /drives/c/Users/pandi/my-repo   haproxy-twilio-port-fix  ./codebuild_build.sh -i amazon/aws-codebuild-local -a /drives/c/Users/pandi/my-repo -s /drives/c/Users/pandi/my-repo/sample-web-app/ Build Command:

docker run -i -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=amazon/aws-codebuild-local" -e "ARTIFACTS=/drives/c/Users/pandi/my-repo" -e "SOURCE=/drives/c/Users/pandi/my-repo/sample-web-app/" -e "INITIATOR=" amazon/aws-codebuild-local:latest

Removing agent-resources_build_1 ... done Removing agent-resources_agent_1 ... done Removing network agent-resources_default Removing volume agent-resources_source_volume Removing volume agent-resources_user_volume Creating network "agent-resources_default" with the default driver Creating volume "agent-resources_source_volume" with local driver Creating volume "agent-resources_user_volume" with local driver Creating agent-resources_agent_1 ... done Creating agent-resources_build_1 ... done Attaching to agent-resources_agent_1, agent-resources_build_1 agent_1 | [Container] 2020/08/25 08:41:32 Waiting for agent ping agent_1 | [Container] 2020/08/25 08:41:33 Waiting for DOWNLOAD_SOURCE agent_1 | [Container] 2020/08/25 08:41:33 Phase is DOWNLOAD_SOURCE agent_1 | [Container] 2020/08/25 08:41:33 CODEBUILD_SRC_DIR=/codebuild/output/src229948805/src agent_1 | [Container] 2020/08/25 08:41:33 Phase complete: DOWNLOAD_SOURCE State: FAILED agent_1 | [Container] 2020/08/25 08:41:33 Phase context status code: YAML_FILE_ERROR Message: YAML file does not exist agent_1 | [Container] 2020/08/25 08:41:33 Runtime error (*clienterr.PhaseContextError: YAML file does not exist) agent-resources_build_1 exited with code 11 Aborting on container exit... ✘

 25/08/2020   14:11.36   /drives/c/Users/pandi/my-repo   haproxy-twilio-port-fix  ls -lrth /drives/c/Users/pandi/my-repo/sample-web-app/ total 9 drwxr-xr-x 1 pandi UsersGrp 0 Aug 25 13:36 src -rwxr-xr-x 1 pandi UsersGrp 3.4K Aug 25 13:36 pom.xml -rw-r--r-- 1 pandi UsersGrp 47 Aug 25 13:36 README.md -rw-r--r-- 1 pandi UsersGrp 11.1K Aug 25 13:36 LICENSE -rwxr-xr-x 1 pandi UsersGrp 161 Aug 25 14:06 buildspec.yml

[hsiddhu2]

Check here , This worked fine for me - https://github.com/aws/aws-codebuild-docker-images/issues/72#issuecomment-450777221