libjxl
libjxl copied to clipboard
libjxl
Build fails with clang
I can't build on Arch Linux libjxl from git with LLVM/Clang instead GCC.
My makepkg.conf:
#!/hint/bash
#
# /etc/makepkg.conf
#
#########################################################################
# SOURCE ACQUISITION
#########################################################################
#
#-- The download utilities that makepkg should use to acquire sources
# Format: 'protocol::agent'
DLAGENTS=('file::/usr/bin/curl -qgC - -o %o %u'
'ftp::/usr/bin/curl -qgfC - --ftp-pasv --retry 3 --retry-delay 3 -o %o %u'
'http::/usr/bin/curl -qgb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
'https::/usr/bin/curl -qgb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
'rsync::/usr/bin/rsync --no-motd -z %u %o'
'scp::/usr/bin/scp -C %u %o')
# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/wget
#-- The package required by makepkg to download VCS sources
# Format: 'protocol::package'
VCSCLIENTS=('bzr::bzr'
'fossil::fossil'
'git::git'
'hg::mercurial'
'svn::subversion')
#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="x86_64"
CHOST="x86_64-pc-linux-gnu"
#-- Compiler and Linker Flags
export CC=clang
export CXX=clang++
#CPPFLAGS=""
CFLAGS="-march=native -mtune=native -O3 -pipe -fno-plt -fexceptions \
-Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security \
-fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fuse-ld=lld"
LTOFLAGS="-flto=auto"
# Rust
CARGO_PROFILE_RELEASE_LTO="thin"
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang
RUSTFLAGS="-C opt-level=3 -C target-cpu=native -C linker=clang -C linker-plugin-lto -C link-arg=-flto -C link-arg=--ld-path=mold"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j$(nproc)"
NINJAFLAGS="-j$(nproc)"
#-- Debugging flags
DEBUG_CFLAGS="-g"
DEBUG_CXXFLAGS="$DEBUG_CFLAGS"
#DEBUG_RUSTFLAGS="-C debuginfo=2"
#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Makepkg defaults: BUILDENV=(!distcc !color !ccache check !sign)
# A negated environment option will do the opposite of the comments below.
#
#-- distcc: Use the Distributed C/C++/ObjC compiler
#-- color: Colorize output messages
#-- ccache: Use ccache to cache compilation
#-- check: Run the check() function if present in the PKGBUILD
#-- sign: Generate PGP signature file
#
BUILDENV=(!distcc color ccache check !sign)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
#DISTCC_HOSTS=""
#
#-- Specify a directory for package building.
#BUILDDIR=/tmp/makepkg
#########################################################################
# GLOBAL PACKAGE OPTIONS
# These are default values for the options=() settings
#########################################################################
#
# Makepkg defaults: OPTIONS=(!strip docs libtool staticlibs emptydirs !zipman !purge !debug !lto)
# A negated option will do the opposite of the comments below.
#
#-- strip: Strip symbols from binaries/libraries
#-- docs: Save doc directories specified by DOC_DIRS
#-- libtool: Leave libtool (.la) files in packages
#-- staticlibs: Leave static library (.a) files in packages
#-- emptydirs: Leave empty directories in packages
#-- zipman: Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge: Remove files specified by PURGE_TARGETS
#-- debug: Add debugging flags as specified in DEBUG_* variables
#-- lto: Add compile flags for building with link time optimization
#
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug !lto)
#-- File integrity checks to use. Valid: md5, sha1, sha224, sha256, sha384, sha512, b2
INTEGRITY_CHECK=(sha256)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)
#-- Directory to store source code in for debug packages
DBGSRCDIR="/usr/src/debug"
#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Log files: specify a fixed directory where all log files will be placed
#LOGDEST=/home/makepkglogs
#-- Packager: name/email of the person or organization building packages
#PACKAGER="John Doe <[email protected]>"
#-- Specify a key to use for package signing
#GPGKEY=""
#########################################################################
# COMPRESSION DEFAULTS
#########################################################################
#
COMPRESSGZ=(gzip -c -f -n)
COMPRESSBZ2=(bzip2 -c -f)
COMPRESSXZ=(xz -c -z -)
COMPRESSZST=(zstd -c -z -q --threads=0 -)
COMPRESSLRZ=(lrzip -q)
COMPRESSLZO=(lzop -q)
COMPRESSZ=(compress -c -f)
COMPRESSLZ4=(lz4 -q)
COMPRESSLZ=(lzip -c -f)
#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
PKGEXT='.pkg.tar.zst'
SRCEXT='.src.tar.gz'
#########################################################################
# OTHER
#########################################################################
#
#-- Command used to run pacman as root, instead of trying sudo and su
#PACMAN_AUTH=()
paru -S libjxl-git give this.
I tried mold instead lld, but it gives the same output.
Environment
- OS: Arch Linux
- Compiler version: Clang 14.0.6
- CPU type: x86_64 AMD Ryzen 7 4800H
Sorry for attaching link to output, but github say it's too long for comment body.
Care to share your cflags?
It seems you're using lto, but can't see anything from the log aside from -ffat-lto-objects being rejected.
It seems that (tools/) binaries does not see libraries. Will check in docker soon.
@eustas
CARCH="x86_64"
CHOST="x86_64-pc-linux-gnu"
#-- Compiler and Linker Flags
export CC=clang
export CXX=clang++
#CPPFLAGS=""
CFLAGS="-march=native -mtune=native -O3 -pipe -fno-plt -fexceptions \
-Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security \
-fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fuse-ld=lld"
If I change in makepkg.conf 3 lines to:
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -Wl,--thinlto-jobs=all -fuse-ld=lld"
LTOFLAGS="-flto=thin"
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug lto)
It fails, but give this as output.
Note: As of the latest commit d178d552887e4a1c38852deb95937a1703238b03, this seems to go away when removing the -ffat-lto-objects flags from CFLAGS and CXXFLAGS.
-ffat-lto-objects specifically affects libjxl_test.c because -Werror is set.
But, other than that, I still can't figure out why I can't link some binaries in tools/. Without the argument, everything seems to work fine. Even copy pasting the raw clang++ compile commands works the same between the two builds.
Okay, I've narrowed it down somewhat: when -ffat-lto-objects is passed to clang, the libraries are missing some necessary symbols, namely all the Jxl* symbols like JxlEncoderSetBasicInfo.
I just have no idea how that happens when clang is supposed to be ignoring the flag.
I seem to have narrowed it down: when -ffat-lto-objects is passed, CMake changes the compiler flags accordingly, removing and adding some options, even when the compiler just ignores it. I'm guessing this ends up breaking the compilation sometimes.
I'm also guessing that -fPIC and -fPIE affect how the shared libraries are built, but I'm not very well versed in C and C++ so don't take my word on that.
In short, I don't think this is a libjxl bug, but a CMake bug.
Here's an example diff output:
diff --color build/build.ninja build-working/build.ninja
61c61
< FLAGS = -ffat-lto-objects -funwind-tables -Xclang -mrelax-all -fno-omit-frame-pointer -Wno-builtin-macro-redefined -Wall -std=c++11
---
> FLAGS = -fno-rtti -funwind-tables -Xclang -mrelax-all -Xclang -mconstructor-aliases -fno-omit-frame-pointer -fPIE -fmacro-prefix-map=/home/louie/dev/libjxl=. -Wno-builtin-macro-redefined -Wall -std=c++11
77c77,78
< FLAGS = -ffat-lto-objects -funwind-tables -Xclang -mrelax-all -fno-omit-frame-pointer
---
> FLAGS = -fno-rtti -funwind-tables -Xclang -mrelax-all -Xclang -mconstructor-aliases -fno-omit-frame-pointer
> LINK_FLAGS = -fPIE -pie
Is this still a problem? Meanwhile we require CMake version 3.16...3.27, see https://github.com/libjxl/libjxl/pull/2987. Please reopen if something needs to be done here.
