zlib
zlib copied to clipboard
madler
zlib v1.2.12 fails to build shared library
When using configure, compiler is not correctly detected as gcc and -fPIC is not added to SFLAGS. This causes relocation issues when linking under Linux.
This is #608 which has a patch in develop (https://github.com/madler/zlib/commit/05796d3d8d5546cf1b4dfe2cd72ab746afae505d).
Using ar
Checking for x86_64-pc-linux-gnu-gcc...
Checking for shared library support...
Building shared library libz.so.1.2.12.1-motley with cc.
Checking for size_t... Yes.
Checking for off64_t... Yes.
Checking for fseeko... Yes.
Checking for strerror... Yes.
Checking for unistd.h... Yes.
Checking for stdarg.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
+ make -j2
cc -ggdb -D_LARGEFILE64_SOURCE=1 -I. -Isrc.d/ -c -o example.o src.d/test/example.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o adler32.o src.d/adler32.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o crc32.o src.d/crc32.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o deflate.o src.d/deflate.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o infback.o src.d/infback.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o inffast.o src.d/inffast.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o inflate.o src.d/inflate.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o inftrees.o src.d/inftrees.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o trees.o src.d/trees.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o zutil.o src.d/zutil.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o compress.o src.d/compress.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o uncompr.o src.d/uncompr.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o gzclose.o src.d/gzclose.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o gzlib.o src.d/gzlib.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o gzread.o src.d/gzread.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -c -o gzwrite.o src.d/gzwrite.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -I. -Isrc.d/ -c -o minigzip.o src.d/test/minigzip.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/adler32.o src.d/adler32.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/crc32.o src.d/crc32.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/deflate.o src.d/deflate.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/infback.o src.d/infback.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/inffast.o src.d/inffast.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/inflate.o src.d/inflate.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/inftrees.o src.d/inftrees.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/trees.o src.d/trees.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/zutil.o src.d/zutil.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/compress.o src.d/compress.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/uncompr.o src.d/uncompr.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/gzclose.o src.d/gzclose.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/gzlib.o src.d/gzlib.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/gzread.o src.d/gzread.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -include zconf.h -DPIC -c -o objs/gzwrite.o src.d/gzwrite.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -I. -Isrc.d/ -D_FILE_OFFSET_BITS=64 -c -o example64.o src.d/test/example.c
cc -ggdb -D_LARGEFILE64_SOURCE=1 -I. -Isrc.d/ -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o src.d/test/minigzip.c
ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o
cc -ggdb -D_LARGEFILE64_SOURCE=1 -o example example.o -L. libz.a
cc -ggdb -D_LARGEFILE64_SOURCE=1 -o minigzip minigzip.o -L. libz.a
cc -shared -ggdb -D_LARGEFILE64_SOURCE=1 -o libz.so.1.2.12.1-motley adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -lc
cc -ggdb -D_LARGEFILE64_SOURCE=1 -o example64 example64.o -L. libz.a
/usr/bin/ld: trees.lo: warning: relocation against `_length_code' in read-only section `.text'
/usr/bin/ld: deflate.lo: relocation R_X86_64_PC32 against symbol `z_errmsg' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:282: libz.so.1.2.12.1-motley] Error 1
make: *** Waiting for unfinished jobs....
I checked all the variables inside configure, and uname is not set correctly (it's set to gnu, not Linux) when CHOST is set to what dpkg-architecture -qDEB_HOST_MULTIARCH returns... This breaks detecting full name of gcc binary and wrong parameters are passed to linker.
For some weird reason, zlib 1.2.11 could manage to find gcc even though full name of the binary wasn't detected.
I am running into similar issues and I had to downgrade back to 1.2.11; many other programs failed to compile when I moved to 1.2.12.
For instance harfbuzz just now:
c++ -o util/hb-view util/hb-view.p/hb-view.cc.o -Wl,--as-needed -Wl,--no-undefined -fPIC '-Wl,-rpath,$ORIGIN/../src' -Wl,-rpath-link,/home/Temp/rbt/harfbuzz-4.2.0/BUILD_DIRECTORY/src -Wl,--start-group src/libharfbuzz.so.0.40200.0 /usr/lib64/libfreetype.so /usr/lib/libcairo.so /usr/lib/libglib-2.0.so -Wl,--end-group
/usr/bin/ld: /usr/lib/libpng16.so.16: undefined reference to `inflateReset2@ZLIB_1.2.3.4'
/usr/bin/ld: /usr/lib/libpng16.so.16: undefined reference to `inflateValidate@ZLIB_1.2.9'
/usr/bin/ld: /usr/lib/libxml2.so.2: undefined reference to `gzopen64@ZLIB_1.2.3.3'
/usr/bin/ld: /usr/lib/libxml2.so.2: undefined reference to `gzdirect@ZLIB_1.2.2.3'
When it doesn't detect the system is Linux, it doesn't set soname or use version map, so the exported symbols don't include symbol versions.
Switching to CMake instead of configure / make at least solves the issue with the relocation issue on rh8.5.
Switching to CMake instead of configure / make at least solves the issue with the relocation issue on rh8.5.
When cross-compiling, using configure is a lot easier as no toolchain file is needed; just set CHOST before running configure. Paths and filenames can differ depending on the operating system version. We use a build matrix with over 10 different compilers and 3 operating systems.
have you tried to use git bisect to find the commit which introduced thebug ? (https://www.git-tower.com/learn/git/faq/git-bisect/)
have you tried to use git bisect to find the commit which introduced thebug ? (https://www.git-tower.com/learn/git/faq/git-bisect/)
Git blame would be a lot faster as I know what line the bug is...
Commit hash is 73014202489f913dbffc91d22089ea8a8920c054 ;)
73014202489f913dbffc91d22089ea8a8920c054 got introduced for v1.2.4.5 more than 10 years ago. I thought the reported issue is a more recent regression of v1.2.12.
73014202489f913dbffc91d22089ea8a8920c054 got introduced for v1.2.4.5 more than 10 years ago. I thought the reported issue is a more recent regression of v1.2.12.
That's the first commit that fails, hence introducing the bug... It had workaround that worked up to v1.2.11, but not in v1.2.12 anymore... I can't track workarounds when going commits backwards as that would require modifying commits after fetching the commit in script and that is not possible in non-interactive shell.
This issue has also affected the Fedora distribution and thus we cannot rebase zlib to its newest 1.2.12 version.
Did you find any workarounds for this yet? @mtl1979
This issue has also affected the Fedora distribution and thus we cannot rebase zlib to its newest 1.2.12 version.
Did you find any workarounds for this yet? @mtl1979
Only workaround I know is to use incorrect CHOST (already canonicalized) and then manually specify CC using full path to compiler... This should work with the tip of develop branch...
@mtl1979 This PR solves it for me: https://github.com/madler/zlib/pull/607
@mtl1979 This PR solves it for me: #607
That PR fixes the issue when not cross-compiling... But when cross-compiling, that PR is not enough like I mentioned above.
Git blame would be a lot faster as I know what line the bug is...
What line is the bug and what is the bug? (That commit has all the changes for that version, so it doesn't help find your line.)
@madler https://github.com/madler/zlib/blob/develop/configure#L35 ... The regex is too simplified... Canonicalizing is the cleanest solution, but it requires copying over the canonicalizing script.
See https://github.com/zlib-ng/zlib-ng/blob/develop/configure#L25
@madler We did normalize the triplet before calling zlib's configure, but that didn't work anymore as the prefix of the compiler binary isn't same as the normalized CHOST. That is why zlib-ng only use the normalized CHOST to deduce the operating system. Not all Ubuntu versions have symbolic link for all relevant binaries from the normalized triplet to original triplet.
@madler Obviously handling of empty CC variable... Like I said earlier, part of that was fixed in 1.2.12.1, but the fix was incomplete and only worked when not cross-compiling. To detect operating system as Linux and finding correct gcc, it needs to use two different triplets as I already mentioned... I tried to just fix the sed regular expression, but that would only fix it for recent Ubuntu versions, not all other operating systems it doesn't detect correctly.
I assume by fixed, you are referring to commit https://github.com/madler/zlib/commit/05796d3d8d5546cf1b4dfe2cd72ab746afae505d . Please explain, with an example, what about the fix is incomplete, and how the result differs from 1.2.11. What is a "triplet"? How were there two different "triplets" in 1.2.11?
@madler Triplet is target for a compiler, for example aarch64-linux-gnu, canonicalized version of that same triplet is aarch64-unknown-linux-gnu. You can test the sed regular expression with both of them to see if it parses them correctly.
As you pointed out, that regular expression hasn't changed since 2011. What did 12.2.11 do that it no longer does, even with commit https://github.com/madler/zlib/commit/05796d3d8d5546cf1b4dfe2cd72ab746afae505d ?
@madler I've already explained the core issue... There is two alternatives, either it uses wrong binaries, targeting host operating system instead of cross-compile target, or it fails during linking shared library due to incorrect linker flags. Specifying CC manually doesn't change the filename for other binaries like it does when using CHOST. It might be combination of a long-standing bug/regression and compiler packagers dropping symbolic links from canonicalized triplet to non-canonicalized triplet.
Obviously using non-canonicalized triplet is the right/preferred choice as then the triplet matches prefix used by the toolchain.
Yes, I know the end result. I'm trying to understand how. Let me ask more explicit questions. In your example, configure is trying to run the executable x86_64-pc-linux-gnu-gcc. Does that executable exist? If so, what does it output with the -v option? If it doesn't exist, what are you expecting to happen?
Yes, I know the end result. I'm trying to understand how. Let me ask more explicit questions. In your example, configure is trying to run the executable
x86_64-pc-linux-gnu-gcc. Does that executable exist? If so, what does it output with the-voption? If it doesn't exist, what are you expecting to happen?
It doesn't exist. Closest existing binary is /usr/bin/x86_64-linux-gnu-gcc.
To answer what should happen, is that it should ignore -unknown-, -pc- etc.