[Experiment] Using substrate to build static lib

[CedricGatay]

Context

I currently work for Acinq and I am trying to use native-image generation to be able to use eclair-core on iOS devices.

To do this, JFXMobile part is not the one we're interested in but we'd rather generate a static library to link against from our iOS project written in Swift.

TLDR This is an issue to share the state of my work regarding using substrate to build static libraries. Scala 2.12 / Akka 2.5 project work, but break as soon as we use cryptographic extensions / https. If there are things that are interesting for this project (or projects around), this is a discussion issue.

Current work

On substrate

I forked substrate to adjust some thing that were blocking me. Work is available on the branch named experiment/akka_scala_eclair.

Adjustments are the following:

• added List<String> nativeBuildOptions to ProjectConfiguration to allow passing flags from external tool to native-image` command related commit
• added buildStaticLib to ProjectConfiguration to allow triggering a static build instead of a full application related commit
• handled linking in the case of buildStaticLib to produce a fat static library with needed symbols from static libs generated from the JDK (like libjvm.a, libjava.a...) related commit
• static build results (.h and .a) is in build/client/${arch}/gvm/shared

On client-gradle-plugin

I also forked client-gradle-plugin to be able to call substrate instead of omega.

I updated the following:

• set a graalPath property to be able to select graal vm to be used and fix build issues when migrating from omega to substrate related commit

• added reflectionConfigFile, buildStaticLib, and nativeBuildOptions to pass arguments to substrate related commit 1 and related commit 2

On client-samples

I forked the samples project and updated the Gradle/HelloFX project, I removed the JavaFX parts to make it a sample using scala 2.12 and akka 2.5 that does a ping / pong available here.

I then use the resulting static library (build/client/arm64-ios/shared/HelloFX.a) in an iOS project available here.

It is working properly !

As a base environment to make this work I have :

• Mac OS Catalina
• llvm 8.0
• GraalVM from Gluon (http://download2.gluonhq.com/substrate/graalvm/graalvm-unknown-java11-19.3.0-dev-gvm-3-${host}.zip from #34 )
• static libs from jdk11-ea+9

Issue

When I enable https support in native-image with --enable-url-protocols=http,https I get missing symbols when linking:

• _JVM_FillInStackTrace is missing
• _Java_jdk_net_MacOSXSocketOptions_* (seems to be in libextnet.a)

Undefined symbols for architecture arm64:
  "_Java_jdk_net_MacOSXSocketOptions_setTcpkeepAliveProbes0", referenced from:
      _MacOSXSocketOptions_setTcpkeepAliveProbes0_667486ab22396e17ac782b9be19cf70e83bdb1fa in hellofx.main.a(llvm.o)
  "_JVM_FillInStackTrace", referenced from:
      _Java_java_lang_Throwable_fillInStackTrace in hellofx.main.a(Throwable.o)
  "_Java_jdk_net_MacOSXSocketOptions_setTcpKeepAliveTime0", referenced from:
      _MacOSXSocketOptions_setTcpKeepAliveTime0_489b5f936900a1d96819ba96c46f154c7da15fe3 in hellofx.main.a(llvm.o)
  "_Java_jdk_net_MacOSXSocketOptions_keepAliveOptionsSupported0", referenced from:
      _MacOSXSocketOptions_keepAliveOptionsSupported0_08c8c171405626502491990f43d89ed2dad5ca1f in hellofx.main.a(llvm.o)
  "_Java_jdk_net_MacOSXSocketOptions_getTcpKeepAliveIntvl0", referenced from:
      _MacOSXSocketOptions_getTcpKeepAliveIntvl0_3c6a1009bb3a39a1a7c77219de7a65f1e30d9750 in hellofx.main.a(llvm.o)
  "_Java_jdk_net_MacOSXSocketOptions_getTcpkeepAliveProbes0", referenced from:
      _MacOSXSocketOptions_getTcpkeepAliveProbes0_661daa644bce3197db58b7b18e2fd176494d028b in hellofx.main.a(llvm.o)
  "_Java_jdk_net_MacOSXSocketOptions_setTcpKeepAliveIntvl0", referenced from:
      _MacOSXSocketOptions_setTcpKeepAliveIntvl0_443e365932509a5c53b5a7690abada6a409d7963 in hellofx.main.a(llvm.o)
  "_Java_jdk_net_MacOSXSocketOptions_getTcpKeepAliveTime0", referenced from:
      _MacOSXSocketOptions_getTcpKeepAliveTime0_26ecd9ad54542e93b061c6254e382d269ef9c6e1 in hellofx.main.a(llvm.o)
ld: symbol(s) not found for architecture arm64

[johanvos]

This might be fixed with the new static libs that we are about to build from openjdk/mobile

[CedricGatay]

I’ve been able to get past this by adding method stubs in a `thread.cˋ file viewable here

[CedricGatay]

Update on experiment

I am currently updating the experiment to depends on latest changes made to Graal / Substrate.

Dependencies

I have updated substrate's current state to allow what's necessary for me (building static library and passing additional flags to native-image command), the fork is available here and installs a binary with the version 0.0.15-ACINQ-SNAPSHOT (so that is does not mess up any existing version) : https://github.com/CedricGatay/substrate/tree/acinq/2020-02

I have also updated gradle plugin to allow passing the new flags properly (same numbering is used) : https://github.com/CedricGatay/client-gradle-plugin

Test project

The test project I use is available https://github.com/CedricGatay/client-samples

It consists on a scala 2.12 / akka 2.5 / eclair framework proof of concept, the goal is to build it targeting iOS (as stated in the initial issue), I use GraalVM from https://download2.gluonhq.com/subtrate/graalvm/graalvm-svm-darwin-20.0.0-ea+22.zip and it ends with a not so nice error during nativeCompile phase, which is "argument list too long". But as far as I know, we can't make macOS have a longer argument list (from the default one 256kb).

For reference, here is the stacktrace, the specified folder contains ~202 000 items (distributed as 67k *.o and 134k *.bc files)

Fatal error: org.graalvm.compiler.debug.GraalError: java.io.IOException: Cannot run program "ld" (in directory "/Users/cgatay/Documents/work/acinq/client-samples/Gradle/HelloFX/build/client/arm64-ios/gvm/tmp/SVM-1581512148236/llvm"): error=7, Argument list too long
        at com.oracle.svm.core.graal.llvm.LLVMNativeImageCodeCache.nativeLink(LLVMNativeImageCodeCache.java:581)
        at com.oracle.svm.core.graal.llvm.LLVMNativeImageCodeCache.linkCompiledBatches(LLVMNativeImageCodeCache.java:455)
        at com.oracle.svm.core.graal.llvm.LLVMNativeImageCodeCache.layoutMethods(LLVMNativeImageCodeCache.java:140)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:604)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:447)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: java.io.IOException: Cannot run program "ld" (in directory "/Users/cgatay/Documents/work/acinq/client-samples/Gradle/HelloFX/build/client/arm64-ios/gvm/tmp/SVM-1581512148236/llvm"): error=7, Argument list too long
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
        at com.oracle.svm.core.graal.llvm.LLVMNativeImageCodeCache.nativeLink(LLVMNativeImageCodeCache.java:570)
        ... 10 more
Caused by: java.io.IOException: error=7, Argument list too long
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
        ... 12 more
Error: Image build request failed with exit status 1

[CedricGatay]

As a follow up, if anyone gets the same issue, the problem was the flag -H:LLVMBatchesPerThread that I had set earlier to deal with llvm compile issues.