Node application missing gyp

[suruaku]

Hi I just found this project and gave it a try, its really nice! While writing this I actually solved my issue... :) So here comes the issue and solution. I checked old issues and could not find similar one.

I build my image from Dockerfile and it works:

FROM node:12-alpine AS builder

WORKDIR /app
COPY ./ ./     # I use .dockerignore so this do not copy redundant files
RUN npm install
RUN npm run build 

FROM node:12-alpine
WORKDIR /app
COPY --from=builder /app ./
EXPOSE 3000
CMD ["npm", "start"]

Then I use docker-slim build my-app:latest and you can see some errors there:

docker-slim[build]: info=http.probe message='using default probe'
docker-slim[build]: state=started
docker-slim[build]: info=params target=my-app:latest continue.mode=probe rt.as.user=true keep.perms=true
docker-slim[build]: state=image.inspection.start
docker-slim[build]: info=image id=sha256:a56997396c98c61e3432ed22a2f701d9adaf4606f2f237f6ab7572de0f73de1e size.bytes=262095118 size.human=262 MB
docker-slim[build]: info=image.stack index=0 name='node:12-alpine' id='sha256:bac10e6a703a46f733e0900af82db4d1148953975891133e590c599c2c8859a5'
docker-slim[build]: info=image.stack index=1 name='my-app:latest' id='sha256:a56997396c98c61e3432ed22a2f701d9adaf4606f2f237f6ab7572de0f73de1e'
docker-slim[build]: info=image.exposed_ports list='3000'
docker-slim[build]: state=image.inspection.done
docker-slim[build]: state=container.inspection.start
docker-slim[build]: info=container status=created name=dockerslimk_2257491_20200601114432 id=2ee1552a348d36c90aa7e27cc8dadaa215ca66590b98cfa66ed761894ececbeb
time="2020-06-01T14:44:32+03:00" level=error msg="channel.Client.Read: read error (read tcp 127.0.0.1:42588->127.0.0.1:32805: read: connection reset by peer), exiting..."
time="2020-06-01T14:44:32+03:00" level=error msg="channel.NewCommandClient: channel verify error = read tcp 127.0.0.1:42588->127.0.0.1:32805: read: connection reset by peer"
docker-slim[build]: info=cmd.startmonitor status=sent
docker-slim[build]: info=event.startmonitor.done status=received
docker-slim[build]: info=container name=dockerslimk_2257491_20200601114432 id=2ee1552a348d36c90aa7e27cc8dadaa215ca66590b98cfa66ed761894ececbeb target.port.list=[32806] target.port.info=[3000/tcp => 0.0.0.0:32806] message='YOU CAN USE THESE PORTS TO INTERACT WITH THE CONTAINER'
docker-slim[build]: state=http.probe.starting message='WAIT FOR HTTP PROBE TO FINISH'
docker-slim[build]: info=continue.after mode=probe message='no input required, execution will resume when HTTP probing is completed'
docker-slim[build]: info=prompt message='waiting for the HTTP probe to finish'
docker-slim[build]: state=http.probe.running
docker-slim[build]: info=http.probe.ports count=1 targets='32806'
docker-slim[build]: info=http.probe.commands count=1 commands='GET /'
docker-slim[build]: info=http.probe.call status=200 method=GET target=http://127.0.0.1:32806/ attempt=1  time=2020-06-01T11:44:49Z
docker-slim[build]: info=http.probe.summary total=1 failures=0 successful=1
docker-slim[build]: state=http.probe.done 
docker-slim[build]: info=event message='HTTP probe is done'
docker-slim[build]: state=container.inspection.finishing
docker-slim[build]: state=container.inspection.artifact.processing
docker-slim[build]: state=container.inspection.done
docker-slim[build]: state=building message='building optimized image'
docker-slim[build]: state=completed
docker-slim[build]: info=results status='MINIFIED BY 4.30X [262095118 (262 MB) => 60929274 (61 MB)]'
docker-slim[build]: info=results  image.name=my-app.slim image.size='61 MB' data=true
docker-slim[build]: info=results  artifacts.location='/home/suruaku/.local/bin/.docker-slim-state/images/a56997396c98c61e3432ed22a2f701d9adaf4606f2f237f6ab7572de0f73de1e/artifacts'
docker-slim[build]: info=results  artifacts.report=creport.json
docker-slim[build]: info=results  artifacts.dockerfile.original=Dockerfile.fat
docker-slim[build]: info=results  artifacts.dockerfile.new=Dockerfile
docker-slim[build]: info=results  artifacts.seccomp=my-app-seccomp.json
docker-slim[build]: info=results  artifacts.apparmor=my-app-apparmor-profile
docker-slim[build]: state=done
docker-slim[build]: info=report file='slim.report.json'

and when I try to run the slim image I get this error:

npm ERR! code MODULE_NOT_FOUND
npm ERR! Cannot find module 'node-gyp/bin/node-gyp'
npm ERR! Require stack:
npm ERR! - /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/noop.js

I followed #149 and tried to build with docker-slim build --include-path=/app my-app but it gave the same error.

Solution: I built the image using docker-slim build --include-path=/app --include-path=/usr/local/lib/node_modules/npm/node_modules my-app:latest

I am not sure if that is the best solution, but it worked. I tried to use --include-path=/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle and --include-path=/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/noop.js but they gave the same error mentioned before. Downside of my solution is that fat image is 262MB and slim is 239MB. So it is still pretty big.

[kcq]

@suruaku the npm-lifecycle dependency should have been picked up and it shouldn't need to be included explicitly (--include-path is a bit low level, so simply including npm-lifecycle/noop.js didn't help because it relies on other things that need to be included too; --include-app-package, which will take care of the extra depdencies, will be a better option, but it's still a WIP).

Either way, I'd like to repro the condition to find the root cause... What does your app do? What do you have listening on port 3000? What's your stack there? What node packages are you using?

[suruaku]

Okay thanks for the info! Here is the whole project I built an image for. The image was for front-end directory in that project. There is update.sh script which uses the exactly command I used to build the image.

The app is small blogging app. Front-end runs at port 3000 and API at port 9000. Stack on 3000 is NuxtJs, VueJS and Vuetify. Node packages.

[kcq]

@suruaku thank you for sharing the app! This is super userful and it'll help with my repro... Vue/Nuxt apps are fun especially when you use the universal mode and you have a bit of extra logic on the server side :-) I'll post an update soon

[kcq]

Managed to repro the condition. Ended up with a couple of different workarounds... The first one is similar to what you have in your shell script, but I only included the global node-gyp dependency for the npm package (no need to include everything). With the other workaround I just got rid of npm start' and used next-start' (which I added to the project after turning all other dependencies into dev depdendencies) to start the server.

Right now i'm investigating how node-gyp is used by npm and why it wasn't picked up...

[ryands17]

@kcq 👋 It works when adding --include-path for node-gyp but in this case, are there any other libs that would need to be added in this case or this is just the exception?

[kcq]

@kcq 👋 It works when adding --include-path for node-gyp but in this case, are there any other libs that would need to be added in this case or this is just the exception?

that should be it and there's actually a fix for it in master, so you won't need to using this --include-path trick with the next release.