docker-alpine-glibc icon indicating copy to clipboard operation
docker-alpine-glibc copied to clipboard

Published 20 hours ago • verified publisher icon Docker-Hub-frolvlad

JDK11 crashed in this base image

Open menghuxiashan opened this issue 4 years ago • 9 comments

Dockerfile:

FROM frolvlad/alpine-glibc:latest
RUN mkdir -p /usr/java/
ADD jdk-11.0.5_linux-x64_bin.tar.gz /usr/java/
ENV JAVA_HOME=/usr/java/jdk-11.0.5
ENV PATH ${PATH}:${JAVA_HOME}/bin

when enter container and input command:

/ # java -version
java version "11.0.5" 2019-10-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.5+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.5+10-LTS, mixed mode)
[thread 11 also had an error]
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f28dcb2aab1, pid=6, tid=21
#
# JRE version: Java(TM) SE Runtime Environment (11.0.5+10) (build 11.0.5+10-LTS)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0.5+10-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libc.so.6+0x11dab1][thread 12 also had an error]

#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h" (or dumping to //core.6)
#
# An error report file with more information is saved as:
# //hs_err_pid6.log
[thread 9 also had an error]
[thread 16 also had an error]
[thread 13 also had an error]
[thread 7 also had an error]
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
Aborted (core dumped)

Why this happened? Can someone help me? Thanks very much!

menghuxiashan avatar Dec 24 '19 09:12 menghuxiashan

I have no idea why this may happen. I can only suggest using OpenJDK compiled for musl libc.

frol avatar Dec 25 '19 09:12 frol

FYI: I am maintaining some company internal docker images based on this Dockerfile and the AdoptOpenJDK Version: https://github.com/AdoptOpenJDK/openjdk-docker/blob/master/11/jdk/alpine/Dockerfile.hotspot.releases.full

In the AdoptOpenJDK Dockerfile, ZLIB and GCC_LIBS are installed. I just run a test without ZLIB nor GCC_LIBS and while JDK8 works well, JDK10, 11 and 13 crashed.

xfh avatar Mar 02 '20 16:03 xfh

I just installed Zulu OpenJDK 11.0.7 JRE into this base image:

ADD https://cdn.azul.com/zulu/bin/zulu11.39.15-ca-jre11.0.7-linux_x64.tar.gz /opt/
ENV JAVA_HOME /opt/zulu11.39.15-ca-jre11.0.7-linux_x64

java --version gains me a fairly similar crash:

openjdk 11.0.7 2020-04-14 LTS
OpenJDK Runtime Environment Zulu11.39+15-CA (build 11.0.7+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.39+15-CA (build 11.0.7+10-LTS, mixed mode)
[thread 79 also had an error]#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fbb5414da41, pid=64, tid=69
#
# JRE version: OpenJDK Runtime Environment (Zulu11.39+15-CA) (11.0.7+10) (build 11.0.7+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.7+10-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libc.so.6+0x11da41]

#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %P" (or dumping to //core.64)
#
# An error report file with more information is saved as:
# //hs_err_pid64.log
[thread 70 also had an error][thread 67 also had an error]

[thread 71 also had an error]
[thread 65 also had an error]
#
# If you would like to submit a bug report, please visit:
#   http://www.azulsystems.com/support/
#
Aborted (core dumped)

This is ldd  $JAVA_HOME/bin/java:

	/lib64/ld-linux-x86-64.so.2 (0x7f0ada2d1000)
	libz.so.1 => /lib/libz.so.1 (0x7f0ada0b4000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f0ada2d1000)
	libjli.so => /opt/zulu11.39.15-ca-jre11.0.7-linux_x64/bin/../lib/jli/libjli.so (0x7f0ad9ea4000)
	libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f0ada2d1000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f0ada2d1000)
Error relocating /opt/zulu11.39.15-ca-jre11.0.7-linux_x64/bin/../lib/jli/libjli.so: __strdup: symbol not found
Error relocating /opt/zulu11.39.15-ca-jre11.0.7-linux_x64/bin/../lib/jli/libjli.so: __rawmemchr: symbol not found

This is sh /usr/glibc-compat/bin/ldd $JAVA_HOME/bin/java:

	linux-vdso.so.1 (0x00007fff74dde000)
	libz.so.1 => /lib/libz.so.1 (0x00007f6aad6d1000)
	libpthread.so.0 => /usr/glibc-compat/lib/libpthread.so.0 (0x00007f6aad6af000)
	libjli.so => /opt/zulu11.39.15-ca-jre11.0.7-linux_x64/bin/../lib/jli/libjli.so (0x00007f6aad49f000)
	libdl.so.2 => /usr/glibc-compat/lib/libdl.so.2 (0x00007f6aad49a000)
	libc.so.6 => /usr/glibc-compat/lib/libc.so.6 (0x00007f6aad2d5000)
	libc.musl-x86_64.so.1 => /lib/libc.musl-x86_64.so.1 (0x00007f6aad240000)
	/lib64/ld-linux-x86-64.so.2 => /usr/glibc-compat/lib64/ld-linux-x86-64.so.2 (0x00007f6aad8f0000)

I guess java still links against musl instead of glibc. Any ideas?

Setting LD_LIBRARY_PATH to /usr/glibc-compat/lib64:/usr/glibc-compat/lib didn't help.

nadavwr avatar Jun 11 '20 17:06 nadavwr

Went over some closed tickets. @frol indicates that ldd output can be misleading here, and suggests prefacing with LD_TRACE_LOADED_OBJECTS=1 instead.

/ # LD_TRACE_LOADED_OBJECTS=1 $JAVA_HOME/bin/java
	linux-vdso.so.1 (0x00007ffe48df2000)
	libz.so.1 => /lib/libz.so.1 (0x00007fe058bf4000)
	libpthread.so.0 => /usr/glibc-compat/lib/libpthread.so.0 (0x00007fe058bd2000)
	libjli.so => /opt/zulu11.39.15-ca-jre11.0.7-linux_x64/bin/../lib/jli/libjli.so (0x00007fe05882b000)
	libdl.so.2 => /usr/glibc-compat/lib/libdl.so.2 (0x00007fe058bcd000)
	libc.so.6 => /usr/glibc-compat/lib/libc.so.6 (0x00007fe058666000)
	libc.musl-x86_64.so.1 => /lib/libc.musl-x86_64.so.1 (0x00007fe058b38000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe058c10000)

Something is obviously working right. I'm a bit surprised to see libc.musl pop up near the end. Looks like it is /lib/libz.so.1 that's pulling in libc.musl. Shouldn't I be able to convince libz to link with glibc?

nadavwr avatar Jun 11 '20 17:06 nadavwr

Try running the binary explicitly with /usr/glibc-compat/lib64/ld-linux-x86-64.so.2:

/usr/glibc-compat/lib64/ld-linux-x86-64.so.2 $JAVA_HOME/bin/java

frol avatar Jun 11 '20 18:06 frol

Same behavior—zlib still leads to musl being loaded:

Ran it with LD_DEBUG=libs to see library loads as they happen. Output for LD_DEBUG=libs /usr/glibc-compat/lib64/ld-linux-x86-64.so.2 $JAVA_HOME/bin/java --version and LD_DEBUG=libs $JAVA_HOME/bin/java --version is identical, so I'm assuming that direct usage of the dynamic linker makes no difference.

# LD_DEBUG=libs /usr/glibc-compat/lib64/ld-linux-x86-64.so.2
$JAVA_HOME/bin/java --version
       214:	find library=libz.so.1 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli/tls/haswell/x86_64:/opt/java/bin/../lib/jli/tls/haswell:/opt/java/bin/../lib/jli/tls/x86_64:/opt/java/bin/../lib/jli/tls:/opt/java/bin/../lib/jli/haswell/x86_64:/opt/java/bin/../lib/jli/haswell:/opt/java/bin/../lib/jli/x86_64:/opt/java/bin/../lib/jli:/opt/java/bin/../lib/tls/haswell/x86_64:/opt/java/bin/../lib/tls/haswell:/opt/java/bin/../lib/tls/x86_64:/opt/java/bin/../lib/tls:/opt/java/bin/../lib/haswell/x86_64:/opt/java/bin/../lib/haswell:/opt/java/bin/../lib/x86_64:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/tls/haswell/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/tls/haswell/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/tls/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/tls/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/haswell/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/haswell/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/jli/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/tls/haswell/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/tls/haswell/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/tls/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/tls/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/haswell/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/haswell/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/x86_64/libz.so.1
       214:	  trying file=/opt/java/bin/../lib/libz.so.1
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/lib/libz.so.1
       214:
       214:	find library=libpthread.so.0 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libpthread.so.0
       214:	  trying file=/opt/java/bin/../lib/libpthread.so.0
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/libpthread.so.0
       214:
       214:	find library=libjli.so [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libjli.so
       214:
       214:	find library=libdl.so.2 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libdl.so.2
       214:	  trying file=/opt/java/bin/../lib/libdl.so.2
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/libdl.so.2
       214:
       214:	find library=libc.so.6 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libc.so.6
       214:	  trying file=/opt/java/bin/../lib/libc.so.6
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/libc.so.6
       214:
       214:	find library=libc.musl-x86_64.so.1 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libc.musl-x86_64.so.1
       214:	  trying file=/opt/java/bin/../lib/libc.musl-x86_64.so.1
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/lib/libc.musl-x86_64.so.1
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/libpthread.so.0
       214:
       214:
       214:	calling init: /lib/libc.musl-x86_64.so.1
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/libc.so.6
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/libdl.so.2
       214:
       214:
       214:	calling init: /lib/libz.so.1
       214:
       214:
       214:	calling init: /opt/java/bin/../lib/jli/libjli.so
       214:
       214:
       214:	initialize program: /opt/java/bin/java
       214:
       214:
       214:	transferring control: /opt/java/bin/java
       214:
       214:	find library=libm.so.6 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libm.so.6
       214:	  trying file=/opt/java/bin/../lib/libm.so.6
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/libm.so.6
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/libm.so.6
       214:
       214:
       214:	calling init: /opt/java/lib/server/libjvm.so
       214:
       214:	find library=librt.so.1 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/librt.so.1
       214:	  trying file=/opt/java/bin/../lib/librt.so.1
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/librt.so.1
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/librt.so.1
       214:
       214:
       214:	calling init: /opt/java/lib/libverify.so
       214:
       214:
       214:	calling init: /opt/java/lib/libjava.so
       214:
       214:	/opt/java/lib/server/libjvm.so: error: symbol lookup error: undefined symbol: JVM_begin_signal_setting (fatal)
       214:	find library=libnss_files.so.2 [0]; searching
       214:	 search path=/opt/java/bin/../lib/jli:/opt/java/bin/../lib		(RPATH from file /opt/java/bin/java)
       214:	  trying file=/opt/java/bin/../lib/jli/libnss_files.so.2
       214:	  trying file=/opt/java/bin/../lib/libnss_files.so.2
       214:	 search cache=/usr/glibc-compat/etc/ld.so.cache
       214:	  trying file=/usr/glibc-compat/lib/libnss_files.so.2
       214:
       214:
       214:	calling init: /usr/glibc-compat/lib/libnss_files.so.2
       214:
       214:
       214:	calling init: /opt/java/lib/libzip.so
       214:
       214:
       214:	calling init: /opt/java/lib/libjimage.so
       214:
       214:	/opt/java/lib/libjava.so: error: symbol lookup error: undefined symbol: Java_java_security_AccessController_doPrivileged (fatal)
       214:	/opt/java/lib/libjava.so: error: symbol lookup error: undefined symbol: Java_jdk_internal_reflect_Reflection_getCallerClass (fatal)
openjdk 11.0.7 2020-04-14 LTS
OpenJDK Runtime Environment Zulu11.39+15-CA (build 11.0.7+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.39+15-CA (build 11.0.7+10-LTS, mixed mode)
[thread 219 also had an error]
#
[thread 220 also had an error]# A fatal error has been detected by the Java Runtime Environment:

#
#  SIGSEGV (0xb) at pc=0x00007f90b3e3da41, pid=214, tid=229
#
[thread 217 also had an error]
# JRE version: OpenJDK Runtime Environment (Zulu11.39+15-CA) (11.0.7+10) (build 11.0.7+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.7+10-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libc.so.6+0x11da41][thread 221 also had an error]

#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /usr/glibc-compat/lib/hs_err_pid214.log
[thread 215 also had an error]
#
# If you would like to submit a bug report, please visit:
#   http://www.azulsystems.com/support/
#
Aborted

nadavwr avatar Jun 18 '20 20:06 nadavwr

We've been able to get the JVM not to crash by lifting a prebuilt glibc-based zlib from another distro (forgot where I saw this, perhaps on a different ticket in this repo). However, we don't feel good about rolling that to production. Still looking for a better solution.

nadavwr avatar Jun 18 '20 20:06 nadavwr

If you do not need a very specific build of JDK, you can install Alpine packages for JDK 8 or 11:

FROM alpine:3.12
RUN apk --no-cache add openjdk8-jre

FROM alpine:3.12
RUN apk --no-cache add openjdk11-jre

prantlf avatar Jul 11 '20 13:07 prantlf

Or just use the official https://hub.docker.com/_/openjdk image, which has Alpine tags.

frol avatar Jul 13 '20 11:07 frol