pack.alpha
pack.alpha copied to clipboard
juxt
GraalVM native-image support
This would allow building to native executable. I'm thinking that we could add a namespace for calling the native-image tool, with suitable defaults (example from jet).
If I did this I'd want it to be really easy, no setting of GRAAL_HOME or whatever. It should just be automatic.
https://www.graalvm.org/sdk/javadoc/ is this what we are after? The native image parts? Not sure.
I didn't actually go as far as checking if there is a Java API available. If such exists, it would be a good match to help with being "automatic" :)
https://medium.com/graalvm/simplifying-native-image-generation-with-maven-plugin-and-embeddable-configuration-d5b283b92f57 suggests the maven plugin is doing it
I have to admit not doing proper background research as there are atleast two tools, https://github.com/luchiniatwork/cambada and https://github.com/taylorwood/clj.native-image, that operate in this area. If native-image has a library api (and is available as a dependency), it would set pack apart from the other tools.
This looks a bit bad 😬 https://github.com/oracle/graal/blob/master/substratevm/src/native-image-maven-plugin/src/main/java/com/oracle/substratevm/NativeImageMojo.java#L166:
Path nativeImageExecutable = getJavaHome().resolve("bin").resolve(withExeSuffix("native-image"));
The "Simplifying native-image generation with Maven plugin and embeddable configuration" post states:
If you are using GraalVM as your JAVA_HOME it is sufficient to just add:
<plugin> <groupId>com.oracle.substratevm</groupId>...
So I guess it relies on the native-image tool being available in the path.
Anyway, thanks for helping on looking into this :)
There's also this following that:
it is also possible to build images even without having a GraalVM release installed on your system. The only requirement is to have a JVMCI enabled JDK on your machine.
Seems like JVMCI is a draft proposal, and only graal is building open jdk with it for now. (These are both statements that need confirming).
I would not be opposed to downloading graalvm into a cache directory and shelling out to that native-image if graalvm cannot be found via the normal methods.
Seems that from 19.00 onwards (see github releases page), native-image was extracted from the Graal distribution and is available as a jar, but it is not independent and relies on the Graal distribution. 300MB of Graal seems like quite a drink in the cache, but might still be worth a try for better usability :) I wonder what is the minimum of deps that the utility actually needs from inside graal...
It's a big cache, but you will need it downloaded either way. Pack can do it for you, or you can do it and specify the location. The cache would be a fallback. Some kind of garbage collection seems worthwhile though.
The idea of tree shaking graal sounds interesting :p.
Thinking out loud:
Maybe we could check for latest release of Graal and even prompt to upgrade to it. A --cache-dir that has a default (say ~/graalvm-cache) would allow users to manage the location of the install. A Graal version could even be selectable via --version, which would default to latest version in the cache directory.
I don't know if native-imagehas different release schedule than graal itself. If so, this might be an additional update step. Initial (and updates) download of native-image could be done via gu utility, so we don't have to perform native-image installation.
My thinking was something along the lines of checking for GRAALVM_HOME, checking if JAVA_HOME contains native-image, then finally creating/updating the cache otherwise.
https://github.com/soc/directories-jvm I'd probably use this for the cache location, because I'm a sucker for compliance.
Doodling thoughts here: It occurred to me that maybe a native-image based AWS Lambda custom runtime could be an spot open in the Clojure cloud tooling :) Pack already has a command for packaging a Lambda for the JVM runtime, but the custom runtime with native-image/SubtrateVM would have better startup time. We could also take care of things like bundling libsunec.so, which would be quite convenient for the user (maybe do deploy also).
OpenJDK11 does support JVMCI behind a flag:
❯ clj -J-XX:+UnlockExperimentalVMOptions -J-XX:+EnableJVMCI
Clojure 1.10.1
user=> (import '[com.sun.management HotSpotDiagnosticMXBean VMOption])com.sun.management.VMOption
user=> (def bean (ManagementFactory/getPlatformMXBean HotSpotDiagnosticMXBean))
WARNING: bean already refers to: #'clojure.core/bean in namespace: user, being replaced by: #'user/bean
Syntax error compiling at (REPL:1:11).
No such namespace: ManagementFactory
user=> (import '[java.lang.management ManagementFactory])java.lang.management.ManagementFactory
user=> (def bean (ManagementFactory/getPlatformMXBean HotSpotDiagnosticMXBean))
#'user/bean
user=> (.getVMOption bean "EnableJVMCI")#object[com.sun.management.VMOption 0xbb9ab64 "VM option: EnableJVMCI value: true origin: VM_CREATION (read-only)"]
user=>
This might be good enough (with a fallback env var for JDK 8 for users who really need it)
Hmm, so in the subtratevm readme, it says
Native image generation is performed by a Java program that runs on JDK 8 with JVMCI.
So how does one get this program, is there a maven library? Seems that I haven't studied this enough :)
The graal team have a custom openjdk8 build with JVMCI in. I doubt the patch will be made upstream by oracle. It might be available in zulu or something though.
7 Dec 2019 16:59:31 Kimmo Koskinen [email protected]:
Hmm, so in the subtratevm readme, it says
Native image generation is performed by a Java program that runs on JDK 8 with JVMCI.
So how does one get this program, is there a maven library? Seems that I haven't studied this enough :)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub [https://github.com/juxt/pack.alpha/issues/47?email_source=notifications&email_token=ABO2DWTWURRPS64BYMFKZG3QXPI7FA5CNFSM4IFWRJOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGGK63Q#issuecomment-562868078] , or unsubscribe [https://github.com/notifications/unsubscribe-auth/ABO2DWTBCHKVR4VKACT6BSLQXPI7FANCNFSM4IFWRJOA] . [https://github.com/notifications/beacon/ABO2DWS4NCQRU7J4XCAY3X3QXPI7FA5CNFSM4IFWRJOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGGK63Q.gif]
The JVMCI approach was later removed, unfortunately. https://github.com/oracle/graal/commit/1ee126205f505405c64ef9e9798efe571bf653e3
It appeared to work by using an artifact like https://mvnrepository.com/artifact/org.graalvm.nativeimage/svm-hosted-native-windows-amd64/22.1.0 and untarring it to a target directory... This might still be viable. The release seems recent still. We already have TDA on the classpath so fetching the correct tar.gz shouldn't be a problem.
One interesting idea might be to leverage the fact that most devs have docker installed, even if they don't have graalvm installed. It would also make it easier to have a reproducible version of native-image that's used per project.