defunct sh and sleep processes when running on slave-jnlp

[DimeOne]

Hello,

When running multiple sh commands within a pipeline, I am receiving multiple defunct sh and sleep processes, if the job runs within a jenkins/jnlp-slave:latest docker container.

The master is running on the same host using jenkins/jenkins:lts-alpine and if the jobs runs on the master, I cannot find any defunct processes.

Setup

• Master container using jenkins/jenkins:lts-alpine
• Slave Agent container using jenkins/jnlp-slave:latest

Repro

The simplest pipeline to reproduce the problem may be:

node() {
  stage ('Run Tests') {
    sh 'echo "hello world"'
  }
    stage ('Run Tests2') {
    sh 'echo "hello world"'
  }
}

And after running the job list the defunct proccesses on the host: $ ps aux | grep defunct | grep -v grep

10000     8054  0.0  0.0      0     0 ?        Z    16:34   0:00 [sh] <defunct>
10000     8057  0.0  0.0      0     0 ?        Z    16:34   0:00 [sleep] <defunct>

[oleg-nenashev]

Likely it is a Pipeline issue, not the container one. Would you be able to try it on a similar environment without containers?

[carlossg]

This could happen if the Java process dies abruptly without reaping children

[DimeOne]

I was not able to reproduce this using a jnlp connected agent running on a ubuntu vm.

The java process did not die in any instance. I guess that is why the master is being started with /bin/tini so that java does not use process id 1.

I will try to create an image where java is not running as PID1 and retry the test.

[DimeOne]

when running the jenkins/jnlp-slave:latest container interactively with bash and then running java does not result in defunct processes, so adding another process should solve it.

https://github.com/jenkinsci/docker/issues/54

[DimeOne]

with tini as PID 1 I am not getting defunct processes, solved that in my image.

ENV TINI_VERSION v0.16.1
RUN wget -O /tini https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini \
    && wget -O /tini.asc https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini.asc \
    && gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 595E85A6B1B4779EA4DAAEC70B588DFF0527A9B7 \
    && gpg --verify /tini.asc \
    && rm /tini.asc \
    && chmod 755 /tini

USER jenkins

ENTRYPOINT ["/tini", "--", "/docker-entrypoint.sh"]