aws-sam-typescript-layers-example icon indicating copy to clipboard operation
aws-sam-typescript-layers-example copied to clipboard

Reduce build time by skipping npm install

Open aclaghorn opened this issue 3 years ago • 3 comments

Only run npm install if the node_modules dir is missing or older than the package.json file. This runs on every function during a sam build, and skipping this step drastically reduces the build time.

aclaghorn avatar Mar 18 '21 23:03 aclaghorn

Thank you very much! I will play with it and merge in a couple of days.

Envek avatar Mar 20 '21 06:03 Envek

It seems that it skips npm install always as all code is being copied by sam build and hence modification time is always is the same.

To try it you can place following commands in Makefile:

build-lambda-common:
	ls -lct > "$(ARTIFACTS_DIR)/ls.out"
	if [ "package-lock.json" -nt "node_modules" ]; then echo "need-to-reinstall" > "$(ARTIFACTS_DIR)/npm.newer"; fi

And see that all files has exactly the same modification timestamp:

-rwxrwxr-x   1 envek envek    826 мар 24 12:16 buildspec.yml
-rw-rw-r--   1 envek envek    113 мар 24 12:16 env.json
-rw-rw-r--   1 envek envek    556 мар 24 12:16 Makefile
drwxrwxr-x 385 envek envek  16384 мар 24 12:16 node_modules
-rw-rw-r--   1 envek envek    970 мар 24 12:16 package.json
-rw-rw-r--   1 envek envek 234841 мар 24 12:16 package-lock.json
-rw-rw-r--   1 envek envek  11420 мар 24 12:16 README.md
drwxrwxr-x   3 envek envek   4096 мар 24 12:16 src
-rw-rw-r--   1 envek envek    322 мар 24 12:16 tsconfig.json
drwxrwxr-x   2 envek envek   4096 мар 24 12:16 events
drwxrwxr-x   4 envek envek   4096 мар 24 12:16 __tests__
drwxrwxr-x   3 envek envek   4096 мар 24 12:16 dist
-rw-rw-r--   1 envek envek    559 мар 24 12:16 samconfig.toml
-rwxrwxr-x   1 envek envek   5991 мар 24 12:16 template.yml

Also, test -nt seems to have portability issues, however it doesn't matter here.

Envek avatar Mar 24 '21 09:03 Envek

I have a Makefile with 65 functions, and I was able to reduce my build time from ~00:17:30 to ~00:13:00 by changing the build-lambda-common to this:

build-lambda-commons:
        cd $(PWD) && [ -d "node_modules" ] || npm ci --cache ~/.npm --prefer-offline
        cd $(PWD) && \
        rm -rf dist && \
        rm -rf tsconfig-only-handler.json && \
        echo "{\"extends\": \"./tsconfig.json\", \"include\": [\"${HANDLER}\"] }" > tsconfig-only-handler.json && \
        npx tsc --build tsconfig-only-handler.json && \
        cp -r dist "$(ARTIFACTS_DIR)/"

Source of inspiration: https://youtu.be/f5W6g_qd-uM?t=3320

samuelmesq avatar Dec 04 '21 16:12 samuelmesq