oracle
[native-image] Cross compilation support?
Any plans to add cross-compilation feature to native-image? I know it's too early to ask for these kind of requirements since we don't even have a working Windows build. IMHO, this would be a nice feature (golang has nice support for it) to have especially if we want to build binaries for all platforms from a CI (Jenkins) machine.
https://medium.com/@chrisgseaton/i-dont-think-we-have-any-immediate-plans-for-cross-compilation-but-it-s-not-a-major-problem-to-bf789384beaa
Definitely would love to see this feature! This would make it so much nicer to use Java for CLI apps and servers that need to be distributed to users on Linux, Mac and Windows.
It would be amazing to have cross-compilation support. Is this still not planned for yet, or is it dependent on the Windows build? It would still be useful to have cross-compilation available to create Mac binaries on a (Linux) CI server, and then add support for Windows cross-compilation later?
@thomaswue @chrisseaton I was just wondering if this is on the roadmap for 2019, or has any ETA etc? I think it would be a game-changer!
@thomaswue @chrisseaton I was just wondering if this is on the roadmap for 2019, or has any ETA etc? I think it would be a game-changer!
Hi @thomaswue @chrisseaton since we're halfway through the year, I just thought I'd check in on this one..
Has there been any development on this issue, or any news/progress to report?
Thanks in advance! 🙂
@thomaswue Now we have java 11 and windows support ready, are there any plans to consider cross-compilation support in the near future?
Adding the support is quite tricky. The issue is that there are parts of the JDK initialized during the native image generation process that could be platform dependent and then end up in the image. One of the alternatives could be to offer a service for creating native images. Also, maybe one could use maybe a system like GitHub actions to build images for different platforms without the need to maintain/setup those machines. Would this help for your use cases or is true cross compilation a requirement?
I do think people should have mentioned this workaround elsewhere.
I am using this https://hub.docker.com/r/oracle/graalvm-ce/ docker image, to build the native-image running on Linux, on my MacBook laptop.
Would this help for your use cases
@thomaswue Thanks. Yeah, GitHub action would work for OSS projects. The issue is, our build system is mostly on Linux and wants to build native-images for windows and mac (mostly for CLI apps). So the cross-compilation would have definitely helped in this scenario.
Yeah it’s not really cross compilation if you need the target platform to build it, that’s just regular old compilation. If Cross compilation is a feature being targeted all the jvm weirdness will need to be solved, I guess the question is, is/when is it a priority?
Here is some discussion on the current limitations - https://graalvm.slack.com/archives/CN9KSFB40/p1582755160011700
Here is some discussion on the current limitations - https://graalvm.slack.com/archives/CN9KSFB40/p1582755160011700
Would it be possible to share a summary of the discussion? The slack archive can only be accessed by people that are member to the workspace.
@truh
Christian Wimmer :
No, cross compilation between different OS is more or less impossible in
the current approach because OS specific JDK classes need to be loaded
in the image generator
The biggest problem is, e.g., the file system and network stack. The JDK code
for that is very different for Linux, MacOS, and Windows. You cannot load the
code from two platforms at the same time, but you need to load the code from
the platform that the image generator runs on.
Anything on cross compiling between architectures? (x86_64 -> arm64, for example)
This is pretty huge for me. Is there any future plan at all to find a way to do this?
+1, it is particularly interesting for arm64 (pi) support where relying on (general) amd64 power to build would be a great enhancement.
@truh
Christian Wimmer : No, cross compilation between different OS is more or less impossible in the current approach because OS specific JDK classes need to be loaded in the image generator ....
WINE is our friend, cross compiling for windows is a big deal. Unfortunately native-image.exe and gu.exe are failing on WINE 6\ WINE7 with 0100:fixme:virtual:NtAllocateVirtualMemoryEx Ignoring 1 extended parameters 0x11fae0
upd. Steps To reproduce:
- Build dockerfile below ``docker build -t graalvm-win .`
- Run it
docker run -it graalvm-win bash -c "wine graalvm-ce-java17-22.0.0.2/lib/installer/bin/gu.exe -h"
FROM ubuntu:21.10
RUN apt-get update \
&& apt-get install -y ca-certificates software-properties-common winbind curl unzip \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
RUN curl --fail --silent --location --retry 3 https://dl.winehq.org/wine-builds/winehq.key | apt-key add - \
&& add-apt-repository 'deb https://dl.winehq.org/wine-builds/ubuntu/ impish main' \
&& dpkg --add-architecture i386 && apt-get update \
&& apt install -y --install-recommends xvfb winehq-devel \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
WORKDIR /opt
ARG GRAALVM_VERSION=22.0.0.2
ARG JAVA_VERSION=java17
ARG GRAALVM_ARCH=windows-amd64
ARG GRAALVM_PKG=https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-$GRAALVM_VERSION/graalvm-ce-$JAVA_VERSION-$GRAALVM_ARCH-$GRAALVM_VERSION.zip
ARG TARGETPLATFORM=linux/amd64
ENV LANG=en_US.UTF-8 \
JAVA_HOME=/opt/graalvm-ce-$JAVA_VERSION-$GRAALVM_VERSION
RUN curl --fail --silent --location --retry 3 ${GRAALVM_PKG} -o /opt/graalvm.zip && unzip -q /opt/graalvm.zip -d /opt && rm /opt/graalvm.zip
RUN wine64 wineboot --init && while pgrep wineserver > /dev/null; do sleep 1; done
CMD java -version
I was expecting the cross-compilation to be possible (when user code is 100% java) by precomputing most of the code-graph in the distro and just importing the result (so, dll, etc included) in native-image main. Can be a saner path than wine which is proven to be quite dependent of the env from my experience.
@rmannibucau it is impossible. You have to have Win build toolchain and Windows build toolchain is not crossplatform, but it works on wine (VS2019 tested). It means we can build container with all tools needed for windows build.
The only blocker is that GraalVM tools ( native-image.exe and gu.exe) don't work with WINE. I will be grateful for any help.
@bademux I understand but it means you generate sources and not .so/.dll directly whereas it is possible to generate .so/.dll directly. That said I'm fine with a cross platform toolchain while it does not rely on wine (once again I know it works on your computer, will likely work on a few others but can trivially break with any upgrade of distro so it is way too fragile for a build chain). My preference would be to rely on light windows docker image finely tagged (versions) so this is reproducible for most people and can even use native windows tools. Maybe @microsoft could help with that?
@rmannibucau it is not entirely true, you pure java code still use platform specific jvm binaries underneath.
2nd - docker is answer for what you call fragility - just freeze you dependencies in container. As well you can use easy-peasy multiplatform docker build with docker buildx, so multiple hardware platforms are covered.
3rd - ms would help here if they allow to redistribute BuildTools by changing license, so you can actually share docker image not a recipe (Dockerfile).
And last but not least and being realistic - it will be easy to solve just one blocker towards build on Windows, then just waiting for "ideal" solution.
@bademux
it is not entirely true, you pure java code still use platform specific jvm binaries underneath.
yes but while the codebase on top of the jvm (understand jre excluded) does not it is fine
Agree on the other point but it should be made easier thanks to graalvm docker images IMHO, currently it is a lot of custom steps and builds and this is what we can simplify a lot IMHO.
So cross-platform seems to be rather impossible. Is there at least any hope for cross-arch compilation, like osx@amd64 -> osx@aarch64?
So cross-platform seems to be rather impossible.
Cross-compilation using native-image is not impossible. We already do it using our own fork of Graal, which allows us to cross architectures on the same OS.
Is there at least any hope for cross-arch compilation, like osx@amd64 -> osx@aarch64?
We already build native images for linux/aarch64 and linux/armv7a on a machine running linux/amd64.
@koutheir , could you please share more info on the cross-compilation? It'd help me a lot. At least in the same OS, different architecture case. Thanks a lot !
We already build native images for linux/aarch64 and linux/armv7a on a machine running linux/amd64.
It is an easy task 😜 to compile multiarch on Linux. It is doable right now without any forks, just use docker multiarch build infrastructure. Give me true cross platform, like compiling static win binary on Linux/osX.
@mixaal, I can't really share how we do it, because it is part of a proprietary product. I simply wanted to inform you that it is possible. I'm sorry.
@bademux, we do real cross-compilation, without any containers or VMs. Doing cross-OS native-image generation is simply the next step on our way if a customer needs that.
@koutheir sounds like big work here. I'm not sure how static binary for win supposed to be cooked on Linux\osX, but finger crossed. Please share!
There is a difference between "it is possible to jump 15m height" and "it is possible to jump to the moon". For now both options is equally impossible until someone invest $.
I can't see the difference if real and imaginary (?) crosscompilation produce the same result.
Did you investigate mingw? It is often used for C/C++ projects so even if it can require to not go directly to .so it can be an option maybe using C/C++ as intermediary language?