Support compile static linux with GraalVM

[ericdallo]

ref

@xerial I could make my Clojure app using sqlite compile with Graalvm for Windows/Linux/MacOS successfully here, but when compiling a Linux static binary, it doesn't work, it seems that the shipped sqlite lib is not static linked, but dynamic linked. Running the static binary I get:

Failed to load native library:sqlite-unknown-5db626ec-518e-4fc0-a5f6-a21818a0c5c8-libsqlitejdbc.so. osinfo: Linux/x86_64
java.lang.UnsatisfiedLinkError: Can't load library: /tmp/sqlite-unknown-5db626ec-518e-4fc0-a5f6-a21818a0c5c8-libsqlitejdbc.so
Exception in thread "main" java.sql.SQLException: Error opening connection
        at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:239)
        at org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:61)
        at org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:28)
        at org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:21)
        at org.sqlite.JDBC.createConnection(JDBC.java:115)
        at org.sqlite.JDBC.connect(JDBC.java:90)
        at java.sql.DriverManager.getConnection(DriverManager.java:677)
        at java.sql.DriverManager.getConnection(DriverManager.java:189)
        at next.jdbc.connection$get_driver_connection.invokeStatic(connection.clj:141)
        at next.jdbc.connection$url_PLUS_etc$reify__747.getConnection(connection.clj:357)
        at next.jdbc.connection$make_connection.invokeStatic(connection.clj:385)
        at next.jdbc.connection$fn__760.invokeStatic(connection.clj:406)
        at next.jdbc.connection$fn__760.invoke(connection.clj:406)
        at next.jdbc.protocols$fn__563$G__558__570.invoke(protocols.clj:24)
        at next.jdbc.connection$fn__764.invokeStatic(connection.clj:415)
        at next.jdbc.connection$fn__764.invoke(connection.clj:406)
        at next.jdbc.protocols$fn__563$G__558__570.invoke(protocols.clj:24)
        at next.jdbc$get_connection.invokeStatic(jdbc.clj:155)
        at sqlite_graalvm_sample.core$insert.invokeStatic(core.clj:15)
        at sqlite_graalvm_sample.core$_main.invokeStatic(core.clj:36)
        at sqlite_graalvm_sample.core$_main.doInvoke(core.clj:36)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at sqlite_graalvm_sample.core.main(Unknown Source)
Caused by: java.lang.Exception: No native library found for os.name=Linux, os.arch=x86_64, paths=[/org/sqlite/native/Linux/x86_64]
        at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:367)
        at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:67)
        at org.sqlite.core.NativeDB.load(NativeDB.java:63)
        at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:235)
        ... 24 more

Is that possible to build this lib statically to make that work?

[xerial]

sqlite-jdbc's native lib for Linux x86_64 is already compiled with a static link flag: https://github.com/xerial/sqlite-jdbc/blob/21055a2b75b70039bd9f688deea2408d7d1c2114/Makefile.common#L77

Two possibilities: this flag was not effective, or other reasons.

[xerial]

We may need to check the .so file with ldd command

[ericdallo]

ldd: warning: you do not have execution permission for `/home/greg/Downloads/libsqlitejdbc.so'
        linux-vdso.so.1 (0x00007fff96564000)
        libc.so.6 => /nix/store/m0xa5bz7vw7p43wi0jppvvi3c9vgqvp7-glibc-2.32-25/lib/libc.so.6 (0x00007fddc0f34000)
        /nix/store/m0xa5bz7vw7p43wi0jppvvi3c9vgqvp7-glibc-2.32-25/lib64/ld-linux-x86-64.so.2 (0x00007fddc13dc000)

[ericdallo]

Any news or suggestions @xerial, ATM this is the only thing that is not working for clojure-lsp: https://github.com/clojure-lsp/clojure-lsp/issues/296

[xerial]

@ericdallo I thought I've been using a version of GCC (xerial/centos5-linux-x86_64-pic) compiled for supporting static link, but it seems not https://github.com/xerial/sqlite-jdbc/blob/master/Makefile#L129

So changing this makefile to use xerial/centos5-linux-x86_64-pic instead of xerial/centos5-linux-x86_64 and running make linux64, which requires Docker, may create sqlite-jdbc that is properly embedding libc etc.

If this still doesn't work, we need to revisit the linker options ..

[ericdallo]

great! so I will try this local, and release a local version of sqlite-jdbc and compile with graalvm and test it :)

[ericdallo]

Hey @xerial I made the docker image changes, generated a local version of sqlite-jdbc and compiled clojure-lsp wit graalvm static flag, finally tested in a ubuntu docker image the static binary and this is still an issue :/ When I try to query the sqlite I get the same error:

Failed to load native library:sqlite-unknown-a3558b77-d78e-45ee-b30a-0a57961f08b0-libsqlitejdbc.so. osinfo: Linux/x86_64  

LMK if need any help testing it

[ericdallo]

Ping @xerial, any news 😔 ?

[xerial]

Sorry for the late response as I don't have so much time for volunteer work 😓

We need to figure out how to embed libc to the binary file (with make native command in Linux) until ldd shows no dynamic link. In my experience of creating statically linked binaries in C++, we needed a custom GCC compiler built with special flags.

[xerial]

@ericdallo If you can find necessary compiler options (in Makefile.common) to produce statically-linked binaries (.so file) in Linux, applying it to sqlite-jdbc should be easy.

[gotson]

I was checking on this why trying to fix the armv6 build, and while i'm no expert at native compilation, i stumbled upon the fact that statically building glibc is discouraged (and should apparently use a specific glibc-static version) and can still end up with dynamically dependencies.

[ericdallo]

Yeah, maybe we should use musl but I have no idea on how or where to start with

[gotson]

I tried building the Alpine version with musl by setting -shared -static but i still end up with a dynamic library 🤷🏻

[gotson]

BTW @ericdallo is this still a concern, even with #631?

[ericdallo]

Yes, #631 will add support to compile with graalvm without clients need to add those configs manually, this issue is that compile graalvm with static flag will not work ATM