yarn/node execution permission denied

[mhermosi]

Hi,

We are using frontend-maven-plugin v1.6 on a Bitbucket pipeline using a custom docker image. this issue has appeared only on that environment if I run the maven build locally it does not fail.

These are part of the maven output:

[INFO] Installing node version v9.4.0
[INFO] Downloading https://nodejs.org/dist/v9.4.0/node-v9.4.0-linux-x64.tar.gz to /root/.m2/repository/com/github/eirslett/node/9.4.0/node-9.4.0-linux-x64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/9.4.0/node-9.4.0-linux-x64.tar.gz into /opt/atlassian/pipelines/agent/build/my-service/web-ui/node/tmp
[INFO] Copying node binary from /opt/atlassian/pipelines/agent/build/my-service/web-ui/node/tmp/node-v9.4.0-linux-x64/bin/node to /opt/atlassian/pipelines/agent/build/my-service/web-ui/node/node
[INFO] Installed node locally.
[INFO] Installing Yarn version v1.8.0
[INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v1.8.0/yarn-v1.8.0.tar.gz to /root/.m2/repository/com/github/eirslett/yarn/1.8.0/yarn-1.8.0.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
Jul 21, 2018 4:47:48 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "Set-Cookie: has_recent_activity=1; path=/; expires=Sat, 21 Jul 2018 17:47:48 -0000". Invalid 'expires' attribute: Sat, 21 Jul 2018 17:47:48 -0000
Jul 21, 2018 4:47:48 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Wed, 21 Jul 2038 16:47:48 -0000; secure; HttpOnly". Invalid 'expires' attribute: Wed, 21 Jul 2038 16:47:48 -0000
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/yarn/1.8.0/yarn-1.8.0.tar.gz into /opt/atlassian/pipelines/agent/build/my-service/web-ui/node/yarn
[INFO] Installed Yarn locally.
[INFO] 
[INFO] --- frontend-maven-plugin:1.6:yarn (yarn install) @ my-service ---
[INFO] Running 'yarn install' in /opt/atlassian/pipelines/agent/build/my-service/web-ui
[ERROR] /opt/atlassian/pipelines/agent/build/my-service/web-ui/node/yarn/dist/bin/yarn: exec: line 32: node: Permission denied

and:

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn install) on project my-service: Failed to run task: 'yarn install' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 126 (Exit value: 126) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :my-service

I am not sure but I think yarn may need user or group execution permission. This is odd because I think alpine linux uses root directly on its docker images. I am using openjdk:8u121-jdk-alpine to build my docker image to be used on bitbucket pipeline.

thanks in advance,

Martin

[mhermosi]

Is this project abandoned?

[tubbynl]

dunnow, i'm having the same issue running from within a Jenkinsfile

[INFO] --- frontend-maven-plugin:1.6:yarn (default-cli) @ nenhkz ---
[INFO] Running 'yarn ' in /var/lib/jenkins/workspace/kz_feature_SPX-450-jhipster-JADFRK7UKVPT6UEHS4NQ2N7P32P64OS7524VZILJY7QQYG4RMUEQ
[ERROR] /var/lib/jenkins/workspace/..../node/yarn/dist/bin/yarn: exec: line 32: node: Permission denied

i've already tried sh "chmod +x node/yarn/dist/bin/yarn"

[mhermosi]

Hi @tubbynl I think problems may be different, at least the cookie thing. Here seems to be an execution permission problem, so maybe developer of this plugin should verify and update execution permission of installed tools like Yarn or Bower etc... I have forked this thing and I will try to add a method to force mode 0777 on INSTALL_DIR/dist/bin/yarn and see how it goes.

=== Update ===

Ok I see the cookie header too on my output.. but I don't see it is affecting the yarn install. because >running locally works. This is happening under container environment in my case.

[mhermosi]

The other problem may be a nodejs permission problem: because the error says on yarn line 32

 1 #!/bin/sh
 2 argv0=$(echo "$0" | sed -e 's,\\,/,g')
 3 basedir=$(dirname "$(readlink "$0" || echo "$argv0")")
 4 
 5 case "$(uname -s)" in 
 6   Linux) basedir=$(dirname "$(readlink -f "$0" || echo "$argv0")");;
 7   *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
 8   *MSYS*) basedir=`cygpath -w "$basedir"`;;
 9 esac
10
11 command_exists() {
12  command -v "$1" >/dev/null 2>&1;
13 }
14
15 if command_exists node; then
16   if [ "$YARN_FORCE_WINPTY" = 1 ] || command_exists winpty && test -t 1; then
17     winpty node "$basedir/yarn.js" "$@"
18   else
19     exec node "$basedir/yarn.js" "$@"
20   fi
21   ret=$?
22 # Debian and Ubuntu use "nodejs" as the name of the binary, not "node", so we
23 # search for that too. See:
24 # https://lists.debian.org/debian-devel-announce/2012/07/msg00002.html
25 # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614907
26 elif command_exists nodejs; then
27   exec nodejs "$basedir/yarn.js" "$@"
28   ret=$?
29 else
30   >&2 echo 'Yarn requires Node.js 4.0 or higher to be installed.'
31   ret=1
32 fi
33 
34 exit $ret

Possible line 19with code exec node "$basedir/yarn.js" "$@" or line 27 where the code is exec nodejs "$basedir/yarn.js" "$@ may be failing and script may be reporting line 32 the last fi

But this seems to be happening only on container execution

[tubbynl]

i ended up creating a build container having maving+yarn (alpine now has a yarn package) and not using the install-node-and-yarn

[mhermosi]

@tubbynl unfortunately for us. for local environment build we need consistency with the tools. so we have to keep using this.

[tubbynl]

for what it's worth then;

• locally on macOS it works
• on Jenkins running in a Jenkinsfile using the maven:3.5.4-alpine image it breaks with the above permission error

The build image i created is

FROM maven:3.5.4-alpine
RUN apk add yarn

[tubbynl]

ok, after fiddling i discovered that the Jenkinsfile build using install-node-and-yarn did succeed if i use maven:3.5.4 (non-alpine) as build image

the fiddling also included adding -u 0 to the docker run arguments o.0, that works but is so-so

[majestica]

@tubbynl can you share your solution?

[narimanmandi]

i have the same issue everything is all right when i run my application in local(mac os) but build stage fails with the same error while trying to build via gitlab ci/cd pipelines @tubbynl do you have any suggestion ?

[mhermosi]

I still have no solution, I just tried building a new Docker image including nodejs and yarn on the system but that did not work given that this plugin is based on installing those binaries.

I thought it was gonna be easy just to use the plugin with the system binaries but is not.

[tubbynl]

my current "solution" is this bitbucket-pipelines.yml file

pipelines:
    default:
    - step:
          name: yarn test
          image: node:10-alpine
          caches:
          - node
          script:
          - printenv
          - yarn install --frozen-lockfile
          - yarn test
    - step:
          name: mvn test
          image: maven:3.5-jdk-8-alpine
          caches:
          - maven
          script:
          - mvn -v
          - mvn -B clean test
    tags:
        '*':
        - step:
              name: yarn build
              image: node:10-alpine
              caches:
              - node
              script:
              - yarn install --frozen-lockfile
              - yarn test
              - yarn build
              artifacts:
              - target/**
        - step:
              name: mvn build
              image: maven:3.5-jdk-8-alpine
              caches:
              - maven
              - node
              script:
              - mvn -B versions:set -DnewVersion=$BITBUCKET_TAG
              - mvn -B package
              artifacts:
              - target/*.war
        - step:
              name: docker
              services:
              - docker
              script:
              - cp -R src/main/docker target/
              - cp target/*$BITBUCKET_TAG.war target/docker/app.war
              - docker version
              - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD $DOCKER_REPO
              - docker build target/docker/ -t $DOCKER_IMAGE:$BITBUCKET_TAG
              - docker push $DOCKER_IMAGE:$BITBUCKET_TAG

essentially; not using this front-end maven plugin when building on the CI; using standard node/maven alpines images to build the frontend and the backend and making sure the front-end build result is correctly included in the backend packaging structure.

[mhermosi]

yeah, i've decided to fork this project and try to see if I can get a solution.. it is simply not making any sense having a permission issue running on bitbucket.. I think basically is missing the execution flag I think let's see where this drives me.

[DanielHons]

@mhermosi did you have any luck? Having the same problem. I am unable to bring my vaadin software in production mode.

[mhermosi]

Hi, I was able to make this work, I am not sure if this may apply to everybody. but debugging my docker image based on alpine I realized that the error is not related with execution permission nor with yarn. it is node distribution itself.

the plugin downloads node distribution which is fine. but in case of alpine linux, not sure if all other Linux images for docker (have not tried) as they are minimal linux distribution, some dependencies (shared libraries) are not available, that is the problem. So how did I overcome this was to create an image including the nodejs / npm distribution from alpine (in my case with node 8.12.0) and match that version with the one used with the plugin. once I did that, the shared libraries were present and the build worked fine.

maybe we should ask the people on node to create a static linked version of their distribution to be used with this library, because finally the plugin itself is not the culprit on this. unfortunately I was not able to find which libraries are missing because alpine does not have ldd and I have not had the time to dig on it to publish a list of missing shared libraries.

The example I saw with pipelines installing yarn and having node-10-alpine I think should work....

[mhermosi]

Oh also be aware that using nodejs on bitbucket (not sure if other CI tools) they set an env var called CI to true. that will mess your node build because CI = true will trigger automatically warning to stop the build as error do.

[DanielHons]

Thank you for youre quick reply. The pipeline mentioned above did not get me much further. By installing node and yarn before i execute the maven plugin i managed to get yarn to work.

But the maven plugin is still looking for the locally installed node and can not find it. Which is kind of strange since i added a symlink to the previous installed node.

This is driving me nuts :/ Unable to bring my application in production mode.

[mhermosi]

it will keep looking for the local installed node, not the system installed one. the sole purpose of installing (on alpine using apk add nodejs and apk install nodejs-npm) is to get the library dependencies

[DanielHons]

Still having problems after installing both it seems.

I receive the following error: Transpilation with gulp has failed: 'gulp build_es6' failed. java.io.IOException: Cannot run program "/opt/atlassian/pipelines/agent/build/target/node/node" (in directory "/opt/atlassian/pipelines/agent/build/target"): error=2, No such file or directory

(Yarn did succeed)

I did install node in my pipeline: default:

• step: image: maven:3.5-jdk-8-alpine name: Build caches:
  • maven script:
  • apk update
  • apk add --update nodejs nodejs-npm && node -v
  • which node
  • npm install -g yarn
  • yarn --version
  • which yarn #- docker info
  • mvn -e -Pproduction verify package # -B batch mode makes Maven less verbose services:
    • docker

And the maven plugin from the last step also states: [INFO] Installed node locally.

[mhermosi]

mmm this means your install is working both node and yarn (yarn used node to run) so rest is another problem with your build, I don't know about Gulp if it is a binary or a node script. another thing is the error itself is not able to find the pipeline agent folder... that looks like something else

[DanielHons]

Yes gulp doesn't matter here. What i do not understand, that the node binary is not found - although node was installed by the plugin.

[dveijck]

@mhermosi en @DonAscon not sure if it is still relevant for you but just posting it anyway for other people who might end up here.

I ran into the same issue and found that there is a similar issue #633 which contains the answer to the problem.

Basically the thing has to do with glibc which is quite a bit of a hassle on Alpine. What fixed it for me was adding:

    apk --no-cache add ca-certificates wget && \
    wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub && \
    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.28-r0/glibc-2.28-r0.apk && \
    apk add glibc-2.28-r0.apk && \
    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.28-r0/glibc-bin-2.28-r0.apk && \
    apk add glibc-bin-2.28-r0.apk && \
    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.28-r0/glibc-i18n-2.28-r0.apk && \
    apk add glibc-i18n-2.28-r0.apk && \

You will likely encounter messages like:

[ERROR] node: /usr/lib/libstdc++.so.6: no version information available (required by node)

but it seems these can be ignored.

It is definitely worth looking at the alpine-pkg-glibc repository as it contains some more detailed information and the latest available releases.

Hope it helps.

[breun]

The issue is that a yarn binary that is built to be used with glibc doesn't work on Alpine out of the box, because Alpine uses musl instead of glibc as its C standard library.

While you can add glibc to Alpine, I think it kind of defeats the point of using Alpine Linux, because it adds so much disk space usage to that otherwise tiny distro. You might be better off using a glibc-based Linux distribution instead (Debian Slim or something).

Or, frontend-maven-plugin needs to be able to download a musl-based Yarn binary, like it already does for NodeJS (see https://github.com/eirslett/frontend-maven-plugin/pull/853).