interesting static compilation issue only when requiring "http/server"

[sam0x17]

So I've tried this in a bunch of the static compilation docker images and on an Ubuntu system on which I have static compilation working. I get the same result everywhere. Basically I can statically compile things unless the crystal file being compiled includes require "http/server", in which case you get the following error:

/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_expand_block':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:239: undefined reference to `inflate'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_compress_block':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:210: undefined reference to `deflate'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_free':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:504: undefined reference to `inflateEnd'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:509: undefined reference to `deflateEnd'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_finish':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:187: undefined reference to `inflateEnd'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:188: undefined reference to `deflateEnd'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_init':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:157: undefined reference to `inflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:168: undefined reference to `deflateInit_'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_flush':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:675: undefined reference to `deflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:680: undefined reference to `zError'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_write':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:596: undefined reference to `deflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:630: undefined reference to `deflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:633: undefined reference to `zError'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_read':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:535: undefined reference to `inflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:546: undefined reference to `inflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:549: undefined reference to `zError'
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/app/gcf'  -rdynamic -static  `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libssl || printf %s '-lssl -lcrypto'` `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libcrypto || printf %s ' -lcrypto'` /usr/lib/libpcre.a /usr/lib/libgc.a /usr/lib/libpthread.a /usr/lib/crystal/core/ext/libcrystal.a /usr/lib/libevent.a /usr/lib/librt.a -L/usr/lib -L/usr/local/lib`

note this issue was previously submitted as #6276 however it turned out to be entirely unrelated to what the original issue implied, so I have opened a new issue here

This isn't a pressing issue for me at all since the binary I was compiling didn't actually end up needing this require statement, however I thought it might help the community if I report it.

[refi64]

Maybe Crystal needs to link with zlib too... What if you pass --link-flags -lz to the linker? That being said, this seems like an issue with the OpenSSL pkg-config file.

[sam0x17]

@kirbyfan64 would I just pass that to crystal build ?

[refi64]

@sam0x17 Yup!

[sam0x17]

@kirbyfan64 interestingly that does not fix it:

$ docker run --rm -it -v $PWD:/app -w /app durosoft/crystal-alpine:latest crystal build src/test.cr --release --static --no-debug --link-flags -lz
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_expand_block':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:239: undefined reference to `inflate'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_compress_block':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:210: undefined reference to `deflate'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_free':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:504: undefined reference to `inflateEnd'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:509: undefined reference to `deflateEnd'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_finish':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:187: undefined reference to `inflateEnd'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:188: undefined reference to `deflateEnd'
/usr/lib/libcrypto.a(c_zlib.o): In function `zlib_stateful_init':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:157: undefined reference to `inflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:168: undefined reference to `deflateInit_'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_flush':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:675: undefined reference to `deflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:680: undefined reference to `zError'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_write':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:596: undefined reference to `deflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:630: undefined reference to `deflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:633: undefined reference to `zError'
/usr/lib/libcrypto.a(c_zlib.o): In function `bio_zlib_read':
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:535: undefined reference to `inflateInit_'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:546: undefined reference to `inflate'
/home/buildozer/aports/main/openssl/src/openssl-1.0.2o/crypto/comp/c_zlib.c:549: undefined reference to `zError'
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/app/gcf' -lz -rdynamic -static  `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libssl || printf %s '-lssl -lcrypto'` `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libcrypto || printf %s ' -lcrypto'` /usr/lib/libpcre.a /usr/lib/libgc.a /usr/lib/libpthread.a /usr/lib/crystal/core/ext/libcrystal.a /usr/lib/libevent.a /usr/lib/librt.a -L/usr/lib -L/usr/local/lib`

[refi64]

Ah, my bad... The -lz needs to somehow end up after the -lcrypto, but I'm not sure how to do that...

[sam0x17]

@kirbyfan64 taking your idea into account, adding --link-flags "-lcrypto -lz" made it compile successfully

[sam0x17]

So to bring everyone up to speed, it looks like the OpenSSL pkg-config needs to be updated so that -lcrypto always goes before -lz. Looking at the code I don't actually see why libz is coming earlier in the list, as http/server.cr requires openssl and THEN http/common.cr, which eventually requires libz. So from what I can tell the requires are ordered correctly in that libcrypto is included before libz.

[rdp]

Was your libz compiled static before you installed openssl? ref: https://groups.google.com/forum/m/#!msg/golang-nuts/Y_gU6kEIICg/Nv_DFfKUCgAJ (bottom). see also #2101 . My hunch is that libcrypto.pc file is broken, probably a bug they should fix upstream but that you can work around as mentioned here...

[stakach]

not sure if this is the best issue to add this too, however I'm seeing some very weird behaviour with static builds This is coming out of STDERR

WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (128)
WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (128)
Failed to raise an exception: END_OF_STACK
[0x458246] ???
[0x42ce58] ???
[0x4395ae] ???
[0x433ec9] ???
[0xcf6348] ???

Tried to raise:: Unknown DW_FORM_data16 (Exception)
  from usr/lib/crystal/core/crystal/dwarf/info.cr:83:29 in '??'
  from usr/lib/crystal/core/crystal/dwarf/info.cr:67:23 in '??'
  from usr/lib/crystal/core/exception/call_stack/elf.cr:10:7 in '??'
  from src/ldso/dl_iterate_phdr.c:45:1 in '??'

That's running an app and making request built using a this Dockerfile: https://github.com/spider-gazelle/spider-gazelle/blob/7c360488ad7b5aef93b92789cd08b8a2ddf03265/Dockerfile

the Unsupported BIO ctrl call reminds me of this issue: #12025

The app runs and handles requests, eventually failing with END_OF_STACK and exiting

[stakach]

I found the warnings occur only when using the edge version of OpenSSL - so that's resolved. So I assume there are few more cases that need to be handled in the near future

WARNING: Unsupported BIO ctrl call (133)
WARNING: Unsupported BIO ctrl call (128)

Still seeing the Tried to raise:: Unknown DW_FORM_data16 (Exception) on static builds