Support building a static share native image library

[ziyilin]

Feature request

Native image can be built as static executable, share library or executable file, but not static share library, as shown in the following table.

	static	dynamic
library	✗	✓
executable	✓	✓

But there are cases that we need to build the Java code into a share library without any dependencies to the outside world. Is there any plan to implement this feature?

Additional context. Add any other context about the feature request here. For example, link to the relevant projects, documentation, standards.

Express whether you'd like to help contributing this feature If this feature is not under development, I can take it over.

[olpaw]

Hi @ziyilin I'm not aware of any near term plans on our side to add such a feature. But I can see how this could be useful.

Feel free to create a PR that implements statically linking of dependencies for native image shared library images.

cc @christianwimmer @vjovanov

[vjovanov]

Adding @gradinac as he is maintaining static linking with Musl and the distroless feature that we have.

[gradinac]

Hey @ziyilin! Would a shared library that only depends on libc work for you? If it does, you can try building your native-image with -H:+StaticExecutableWithDynamicLibC. This option has so far only been used for dynamic executables, but it could also work for your use case. Let us know if you hit any problems with it

[ziyilin]

Thanks. Currently for a simple demo the libc is enough, but more dependencies may be required in the future. But building with -H:+StaticExecutableWithDynamicLibC is for executable file, in my case, I'd like to have a library. Should I provide both @CEntryPoint for library APIs and main for executable? That sounds weird.

[gradinac]

Ah, by only depending on libc, I mean the resulting shared library will be statically linked against all other libraries, except for the libc ones. libc libraries refer to the set of libraries that constitute the system's libc, you can see the list here: https://github.com/oracle/graal/blob/2004ee59b0eec60e2b7a83ee006a2827b7a9a533/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeBootImageViaCC.java#L94

The StaticExecutableWithDynamicLibC option tells native-image to instruct the linker to statically link the necessary libraries. While the name refers to a static executable, the same principle could be used when building a dynamic library.

However, currently, the option also sets the --static flag. Could you delete the following line: https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/SubstrateOptions.java#L83, rebuild GraalVM and create a shared library with StaticExecutableWithDynamicLibC?

For a helloworld shared library, we have the following results: Without -H:+StaticExecutableWithDynamicLibC:

$ ldd test.so
	linux-vdso.so.1 (0x00007fffff1f6000) # automatically mapped in by the kernel
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9ea8f6b000) # libc related library
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9ea8f65000) # libc related library
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f9ea8f49000) # non-libc related library
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9ea8d57000) # libc related library
	/lib64/ld-linux-x86-64.so.2 (0x00007f9ea9845000) # dynamic linker

With -H:+StaticExecutableWithDynamicLibC:

$ ldd test.so
	linux-vdso.so.1 (0x00007ffe714a2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3947e18000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3947e12000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3947c20000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f39486fc000)

[ziyilin]

Thanks, I have managed to compile a shared library with StaticExecutableWithDynamicLibC. But still need to further include libc into the built out shared library in our scenario.

[gradinac]

I'm glad that worked! :smile: Hmm, for your use-case, you'd also like to have libc statically linked to your shared library? Is your shared library used on a glibc based platform? If yes, statically linking against glibc can be quite problematic for a couple of reasons:

• Without a custom glibc build, you'd still have dynamic dependencies at runtime:

warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

• You may encounter strange bugs when statically linking against glibc: https://sourceware.org/bugzilla/show_bug.cgi?id=10652

We have introduced musl support for that exact reason, but we currently only support musl static native-image executables. However, if your target platform uses glibc, I don't think you'd be able to use a dynamic library linked against musl - glibc and musl require separate dynamic linkers.

[begelundmuller]

Hello all! I see it's been a while since this issue was created, so I wanted to ask if this has been solved or has a known workaround?

Our use case is that we've started a project where we're compiling a Java library to a native shared library (.so on Linux) using GraalVM, which so far has been very successful. However, bundling the shared library alongside our main binary (a Go program) is a headache, and we would prefer to distribute a single statically linked binary. To accomplish that, it's my understanding we need a static library (.a file on Linux) instead.

In terms of workarounds, I did find this old comment from @olpaw, but I'm not sure it applies to this case – at least I haven't been able to make it work.

If there isn't a known workaround, we might be interested in contributing a solution here. Unfortunately, I'm not too familiar with GraalVM internals, so any pointers would be appreciated!

[ziyilin]

We didn't go further in this topic due to change of our own project requirement. But there is a workaround in 3 steps: 1. Disable the libc support checking; 2. Output the relocatable file (.o) as native-image compiling result; 3. Statically link the .o file with GCC by yourself. We managed to do this with musl to statically link the output .o file with other static libraries (.a) into a shared library (.so) . We didn't test this workaround in your use case, you can have a try.

The following java source will do the first two steps. Compile it to class file, and add it to native-image's classpath to take effect.

import com.oracle.svm.core.annotate.AutomaticFeature;
import com.oracle.svm.core.c.libc.LibCBase;
import com.oracle.svm.core.c.libc.TemporaryBuildDirectoryProvider;
import com.oracle.svm.core.posix.linux.libc.LibCFeature;
import com.oracle.svm.core.posix.linux.libc.MuslLibC;
import com.oracle.svm.core.util.UserError;
import com.oracle.svm.core.util.VMError;
import com.oracle.svm.hosted.FeatureImpl;
import com.oracle.svm.hosted.NativeImageGenerator;
import org.graalvm.compiler.serviceprovider.JavaVersionUtil;
import org.graalvm.nativeimage.ImageSingletons;

import java.io.IOException;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.StandardCopyOption;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ServiceLoader;

@AutomaticFeature
public class StaticLibcFeature extends LibCFeature {

    @Override
    public void afterRegistration(AfterRegistrationAccess access) {
        String targetLibC = LibCOptions.UseLibC.getValue();
        ServiceLoader<LibCBase> loader = ServiceLoader.load(LibCBase.class);
        for (LibCBase libc : loader) {
            if (libc.getName().equals(targetLibC)) {
                    if (JavaVersionUtil.JAVA_SPEC < 11) {
                        throw UserError.abort("Musl can only be used with labsjdk 11+.");
                    }
                ImageSingletons.add(LibCBase.class, libc);
                return;
            }
        }
        throw UserError.abort("Unknown libc %s selected. Please use one of the available libc implementations.", targetLibC);
    }

     /**
     * Copy the relocatable file and header file from temporary directory to output path.
     */
    @Override
    public void afterImageWrite(AfterImageWriteAccess access) {
        FeatureImpl.AfterImageWriteAccessImpl a = (FeatureImpl.AfterImageWriteAccessImpl) access;
        Path outputDirectory = NativeImageGenerator.generatedFiles(a.getUniverse().getBigBang().getOptions());
        Path tempDirectory = ImageSingletons.lookup(TemporaryBuildDirectoryProvider.class).getTemporaryBuildDirectory();
        try {
            if (Files.notExists(outputDirectory)) {
                Files.createDirectory(outputDirectory);
            }
            Files.walkFileTree(tempDirectory, new SimpleFileVisitor<>() {
                @Override
                public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
                    String fileName = file.getFileName().toString();
                    if (fileName.endsWith(".o") || fileName.endsWith(".h")) {
                        Path target = outputDirectory.resolve(fileName).toAbsolutePath();
                        Files.copy(file, target, StandardCopyOption.REPLACE_EXISTING);
                    }
                    return FileVisitResult.CONTINUE;
                }
            });
        } catch (IOException e) {
            VMError.shouldNotReachHere("Fail to copy file from temporary", e);
        }
    }
}

[begelundmuller]

@ziyilin Thank you for sharing this, it certainly seems like a promising workaround for our use case! Do you have a sense of whether this would also work on Mac/Windows?

Edit: Another question as I'm tinkering with this – in your step 3, do you recall which other libraries you had to link in to make it work (in particular, did you manually have to link any of the Java static libraries)?

[ziyilin]

Do you have a sense of whether this would also work on Mac/Windows?

I think it should work.

did you manually have to link any of the Java static libraries

Yes, I also linked Java static libraries, which are shipped with GraalVM.

[CGDogan]

Dear @begelundmuller , sorry to ping this issue a long time later. Have you been using this script or something similar? It seems that the script by @ziyilin (thank you for providing us this ziyilin) won't compile with the latest Graal SDK. I was wondering if you have a more recent copy. Many thanks and regards

[ennerf]

I ran into the same issue and managed to work around it by intercepting and replacing the compiler arguments, i.e., run native-image with --native-compiler-path=${pathToScript}.sh. I still need to validate some of the results, but here is what I have so far:

Linux (Ubuntu)

#!/bin/bash
# This script intercepts compiler arguments from graalvm native shared images and can replace
# select dynamically linked libraries with statically linked ones. The script was tested with
# GraalVM jdk21.0.1+12.1 on Ubuntu and may need to be modified for other versions.
#
# Use with --native-compiler-path=${pathToThisScript}.sh

# Determine the project name and output path based on the output .so argument
for arg in "$@"; do
  if [[ "$arg" == *.so ]]; then
    OUTPUT_PATH=$(dirname "$arg")
    LIB_NAME=$(basename "${arg%.so}")
    break
  fi
done

# Do a simple forward for any calls that are used to compile individual C files
if [[ -z $LIB_NAME ]]; then
    gcc $*
    exit 0
fi

# Create a debug log in $output/logs
LOG_PATH="${OUTPUT_PATH}/logs"
LOG_FILE="${LOG_PATH}/compiler_commands.txt"
mkdir -p $LOG_PATH

WORKINGDIR=${PWD}
echo "Working directory: ${WORKINGDIR}" > ${LOG_FILE}
echo "Output path: ${OUTPUT_PATH}" >> ${LOG_FILE}
echo "Library name: ${LIB_NAME}" >> ${LOG_FILE}

echo "=====================================================" >> ${LOG_FILE}
echo "                    ORIGINAL ARGS                    " >> ${LOG_FILE}
echo "=====================================================" >> ${LOG_FILE}
echo "$*" >> ${LOG_FILE}

echo "=====================================================" >> ${LOG_FILE}
echo "           SHARED LIBRARY WITH STATIC LIBS           " >> ${LOG_FILE}
echo "=====================================================" >> ${LOG_FILE}
# Path to the system library files
ARCH=$(uname -m)
STATIC_LIBS_PATH="/lib/${ARCH}-linux-gnu"

# Do the original call, but replace dynamic libs with static versions
GCC_ARGS=""
for arg in "$@"
do
    if [ "$arg" = "-lz" ]
    then
        GCC_ARGS+=" ${STATIC_LIBS_PATH}/libz.a"
    else
        GCC_ARGS+=" $arg"
    fi
done

echo "gcc $GCC_ARGS" >> ${LOG_FILE}
gcc $GCC_ARGS

echo "=====================================================" >> ${LOG_FILE}
echo "                   STATIC LIBRARY                    " >> ${LOG_FILE}
echo "=====================================================" >> ${LOG_FILE}
# To create a single static library on linux we need to call 'ar -r' on all .o files.
# In order to also include all static library dependencies, we can first extract the
# .o files and then include them as well.
echo "======= Source archives"  >> ${LOG_FILE}
OBJECTS=${OUTPUT_PATH}/objects
rm -rf ${OBJECTS}
mkdir ${OBJECTS}
AR_ARGS="-rcs ${OUTPUT_PATH}/${LIB_NAME}.a ${OBJECTS}/*.o"
for arg in $GCC_ARGS
do
    if [[ $arg =~ .*\.(a)$ ]]; then
        # extract the objects (.o) of each archive (.a) into
        # separate directories to avoid naming collisions
        echo "$arg"  >> ${LOG_FILE}
        ARCHIVE_DIR=${OBJECTS}/$(basename "${arg%.a}")
        mkdir ${ARCHIVE_DIR}
        cp $arg ${ARCHIVE_DIR}
        cd ${ARCHIVE_DIR}
        ar -x $arg
        cd ${WORKINGDIR}
        AR_ARGS+=" ${ARCHIVE_DIR}/*.o"
    fi
    if [[ $arg =~ .*\.(o)$ ]]; then
        cp $arg ${OBJECTS}
    fi
done

echo "======= Objects"  >> ${LOG_FILE}
find ${OBJECTS} -name "*.o" >> ${LOG_FILE}

echo "======= Archive command"  >> ${LOG_FILE}
echo "ar $AR_ARGS" >> ${LOG_FILE}
ar $AR_ARGS

macOS

#!/bin/bash
# This script intercepts compiler arguments from graalvm native shared images
# and additionally generates a static library. The script was tested with
# GraalVM jdk21+35.1 on macOS and may need to be modified for other versions.
#
# Use with --native-compiler-path=${pathToThisScript}

# Determine the project name and output path based on the output .dylib argument
for arg in "$@"; do
  if [[ "$arg" == *.dylib ]]; then
    OUTPUT_PATH=$(dirname "$arg")
    LIB_NAME=$(basename "${arg%.dylib}")
    break
  fi
done

# Do a simple forward for any calls that are used to compile individual C files
if [[ -z $LIB_NAME ]]; then
    cc $*
    exit 0
fi

# Create a debug log in $output/logs
LOG_PATH="${OUTPUT_PATH}/logs"
LOG_FILE="${LOG_PATH}/compiler_commands.txt"
mkdir -p $LOG_PATH

WORKINGDIR=${PWD}
echo "Working directory: ${WORKINGDIR}" > ${LOG_FILE}
echo "Output path: ${OUTPUT_PATH}" >> ${LOG_FILE}
echo "Library name: ${LIB_NAME}" >> ${LOG_FILE}

echo "=====================================================" >> ${LOG_FILE}
echo "                  SHARED LIBRARY                     " >> ${LOG_FILE}
echo "=====================================================" >> ${LOG_FILE}
# Modify the arguments if needed
CC_ARGS=$*
echo "cc $CC_ARGS" >> ${LOG_FILE}
cc $CC_ARGS

echo "=====================================================" >> ${LOG_FILE}
echo "                   STATIC LIBRARY                    " >> ${LOG_FILE}
echo "=====================================================" >> ${LOG_FILE}
# To create a single static library on macos we need to call 'ar -r' on all .o files.
# In order to also include all static library dependencies, we can first extract the
# .o files and then include them as well.
echo "======= Source archives"  >> ${LOG_FILE}
OBJECTS=${OUTPUT_PATH}/objects
rm -rf ${OBJECTS}
mkdir ${OBJECTS}
AR_ARGS="-rcs ${OUTPUT_PATH}/${LIB_NAME}.a ${OBJECTS}/*.o"
for arg in $*
do
    if [[ $arg =~ .*\.(a)$ ]]; then
        # extract the objects (.o) of each archive (.a) into
        # separate directories to avoid naming collisions
        echo "$arg"  >> ${LOG_FILE}
        ARCHIVE_DIR=${OBJECTS}/$(basename "${arg%.a}")
        mkdir ${ARCHIVE_DIR}
        cp $arg ${ARCHIVE_DIR}
        cd ${ARCHIVE_DIR}
        ar -x $arg
        cd ${WORKINGDIR}
        AR_ARGS+=" ${ARCHIVE_DIR}/*.o"
    fi
    if [[ $arg =~ .*\.(o)$ ]]; then
        cp $arg ${OBJECTS}
    fi
done

echo "======= Objects"  >> ${LOG_FILE}
find ${OBJECTS} -name "*.o" >> ${LOG_FILE}

echo "======= Archive command"  >> ${LOG_FILE}
echo "ar $AR_ARGS" >> ${LOG_FILE}
ar $AR_ARGS

Windows

@echo off
setlocal EnableDelayedExpansion

REM This script intercepts compiler arguments generated by GraalVM's native-image for
REM creating a shared library, and generates both shared and static libraries. The
REM output path has to be specified relative to this script because GraalVM builds
REM in a local temp directory and does not specify the output path anywhere. Tested on
REM GraalVM jdk21.+35.1 on Windows 10 and may need to be modified for other versions.
REM
REM Use with --native-compiler-path=${pathToThisScript}.bat
set OUTPUT_PATH=%~dp0\..\..\target\image

REM Determine the library name based on the .dll argument. ~nP returns the filename
REM without quotes, i.e., "path\myLibrary.dll" returns myLibrary
set LIB_NAME=
for %%P in (%*) do (
    echo %%P | findstr /C:"\.dll" 1>nul
    if !errorlevel!==0 (
        set LIB_NAME=%%~nP
    )
)

REM Do a simple forward for any calls that are used to compile individual C files
IF "%LIB_NAME%"=="" (
    cmd /c cl %*
    exit /b
)

REM Setup log path and log file
set LOG_PATH=%OUTPUT_PATH%\logs
set LOG_FILE=%LOG_PATH%\compiler_commands.txt
if not exist %LOG_PATH% mkdir %LOG_PATH%

echo Working directory: %CD% > %LOG_FILE%
echo Output path: %OUTPUT_PATH% >> %LOG_FILE%
echo Library name: %LIB_NAME% >> %LOG_FILE%

echo ===================================================== >> %LOG_FILE%
echo                   SHARED LIBRARY                      >> %LOG_FILE%
echo ===================================================== >> %LOG_FILE%
REM Modify the arguments if needed
set CL_ARGS=%*
echo cl.exe %CL_ARGS% >> %LOG_FILE%
cmd /c cl.exe %CL_ARGS%

echo ===================================================== >> %LOG_FILE%
echo                   STATIC LIBRARY                      >> %LOG_FILE%
echo ===================================================== >> %LOG_FILE%
REM To create a static library on Windows we need to call lib.exe input.obj /OUT:output.lib
REM We don't want to overwrite the .lib needed to compile against the .dll, so
REM we append "_s" to indicate that it is a static library.
if not exist %OUTPUT_PATH% mkdir %OUTPUT_PATH%
set LIB_ARGS=%LIB_NAME%.obj /OUT:%OUTPUT_PATH%\%LIB_NAME%_s.lib
echo lib.exe %LIB_ARGS% >> %LOG_FILE%
cmd /c lib.exe %LIB_ARGS%

[CGDogan]

thanks

[Mis012]

We didn't go further in this topic due to change of our own project requirement. But there is a workaround in 3 steps: 1. Disable the libc support checking; 2. Output the relocatable file (.o) as native-image compiling result; 3. Statically link the .o file with GCC by yourself. We managed to do this with musl to statically link the output .o file with other static libraries (.a) into a shared library (.so) . We didn't test this workaround in your use case, you can have a try.

The following java source will do the first two steps. Compile it to class file, and add it to native-image's classpath to take effect.

import com.oracle.svm.core.annotate.AutomaticFeature;
import com.oracle.svm.core.c.libc.LibCBase;
import com.oracle.svm.core.c.libc.TemporaryBuildDirectoryProvider;
import com.oracle.svm.core.posix.linux.libc.LibCFeature;
import com.oracle.svm.core.posix.linux.libc.MuslLibC;
import com.oracle.svm.core.util.UserError;
import com.oracle.svm.core.util.VMError;
import com.oracle.svm.hosted.FeatureImpl;
import com.oracle.svm.hosted.NativeImageGenerator;
import org.graalvm.compiler.serviceprovider.JavaVersionUtil;
import org.graalvm.nativeimage.ImageSingletons;

import java.io.IOException;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.StandardCopyOption;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ServiceLoader;

@AutomaticFeature
public class StaticLibcFeature extends LibCFeature {

    @Override
    public void afterRegistration(AfterRegistrationAccess access) {
        String targetLibC = LibCOptions.UseLibC.getValue();
        ServiceLoader<LibCBase> loader = ServiceLoader.load(LibCBase.class);
        for (LibCBase libc : loader) {
            if (libc.getName().equals(targetLibC)) {
                    if (JavaVersionUtil.JAVA_SPEC < 11) {
                        throw UserError.abort("Musl can only be used with labsjdk 11+.");
                    }
                ImageSingletons.add(LibCBase.class, libc);
                return;
            }
        }
        throw UserError.abort("Unknown libc %s selected. Please use one of the available libc implementations.", targetLibC);
    }

     /**
     * Copy the relocatable file and header file from temporary directory to output path.
     */
    @Override
    public void afterImageWrite(AfterImageWriteAccess access) {
        FeatureImpl.AfterImageWriteAccessImpl a = (FeatureImpl.AfterImageWriteAccessImpl) access;
        Path outputDirectory = NativeImageGenerator.generatedFiles(a.getUniverse().getBigBang().getOptions());
        Path tempDirectory = ImageSingletons.lookup(TemporaryBuildDirectoryProvider.class).getTemporaryBuildDirectory();
        try {
            if (Files.notExists(outputDirectory)) {
                Files.createDirectory(outputDirectory);
            }
            Files.walkFileTree(tempDirectory, new SimpleFileVisitor<>() {
                @Override
                public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
                    String fileName = file.getFileName().toString();
                    if (fileName.endsWith(".o") || fileName.endsWith(".h")) {
                        Path target = outputDirectory.resolve(fileName).toAbsolutePath();
                        Files.copy(file, target, StandardCopyOption.REPLACE_EXISTING);
                    }
                    return FileVisitResult.CONTINUE;
                }
            });
        } catch (IOException e) {
            VMError.shouldNotReachHere("Fail to copy file from temporary", e);
        }
    }
}

I think this would make the most sense - have an option to output a .a archive with just the single .o file instead of a finished .so file. The user will be expected to know what to do with that for whatever usecase they have, and it's much simpler and more flexible than other approaches. Maybe the documentation could have an example gcc/clang commandline for manually creating a .so from the .a (the same way it's done by native-image for .so output) as a starting point. This cmdline could probably even be printed when the .a output is created in case there are dynamic parts that would be annoying to have to recreate.