keepalived
keepalived copied to clipboard
acassen
Make keepalived cross compile reproducible
Is your feature request to resolve a problem or provide enhanced functionality? Please describe. Make keepalived cross compile reproducible.
Describe the solution you would like
The output of keepalived -v gives me the configure options, which don't have to be the same for another cross compile system which does recompile the software. I would suggest that these things do not belong in the compilation, as they vary from machine to machine and should not to get printet. I don't know if that's all there is to do, to make keepalived reproducible.
Describe alternatives you have considered It depends on whether there is other information that can vary from build to build with the same version.
Would the feature request be of benefit only to you, or is it more generally applicable? This makes the keepalived reproducible
Keepalived version
keepalived --version
Keepalived v2.2.7 (01/16,2022)
Copyright(C) 2001-2022 Alexandre Cassen, <[email protected]>
Built with kernel headers for Linux 5.10.139
Running on Linux 5.10.139 #0 SMP Thu Sep 8 09:10:35 2022
Distro: openwrt
configure options: --target=x86_64-openwrt-linux --host=x86_64-openwrt-linux --build=x86_64-pc-linux-gnu --program-prefix= --program-suffix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/man --infodir=/usr/info --disable-nls --with-init=SYSV --disable-track-process --runstatedir=/var/run --enable-sha1 --disable-libipset-dynamic --enable-json build_alias=x86_64-pc-linux-gnu host_alias=x86_64-openwrt-linux target_alias=x86_64-openwrt-linux PKG_CONFIG=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/host/bin/pkg-config PKG_CONFIG_PATH=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/lib/pkgconfig:/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/share/pkgconfig PKG_CONFIG_LIBDIR=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/lib/pkgconfig:/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/share/pkgconfig CC=x86_64-openwrt-linux-musl-gcc CFLAGS=-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/keepalived-2.2.7=keepalived-2.2.7 -Wformat -Werror=format-security -DPIC -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/linux-x86_64/linux-5.10.139 LDFLAGS=-L/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-11.3.0_musl/usr/lib -L/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-11.3.0_musl/lib -DPIC -fpic -specs=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/include/hardened-ld-pie.specs -znow -zrelro CPPFLAGS=-I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-11.3.0_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-11.3.0_musl/include/fortify -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-11.3.0_musl/include
Config options: LIBIPSET NFTABLES LVS VRRP VRRP_AUTH VRRP_VMAC JSON DISABLE_TRACK_PROCESS OLD_CHKSUM_COMPAT INIT=SYSV
System options: VSYSLOG MEMFD_CREATE IPV6_MULTICAST_ALL LIBKMOD IPV4_DEVCONF LIBNL3 RTA_ENCAP RTA_EXPIRES RTA_NEWDST RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_SUPPRESS_IFGROUP FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTEXT_FILTER_SKIP_STATS FRA_L3MDEV FRA_UID_RANGE RTAX_FASTOPEN_NO_COOKIE RTA_VIA FRA_PROTOCOL FRA_IP_PROTO FRA_SPORT_RANGE FRA_DPORT_RANGE RTA_TTL_PROPAGATE IFA_FLAGS LWTUNNEL_ENCAP_MPLS LWTUNNEL_ENCAP_ILA IPTABLES NET_LINUX_IF_H_COLLISION NETINET_LINUX_IF_ETHER_H_COLLISION LIBIPVS_NETLINK IPVS_DEST_ATTR_ADDR_FAMILY IPVS_SYNCD_ATTRIBUTES IPVS_64BIT_STATS IPVS_TUN_TYPE IPVS_TUN_CSUM IPVS_TUN_GRE VRRP_IPVLAN IFLA_LINK_NETNSID INET6_ADDR_GEN_MODE VRF SO_MAR
If different configure options are chosen, then a different executable will be produced, since it will contain different options. My understanding is, therefore, that if different configure options are selected, the build will not be reproducible, in the sense that the binaries will be different.
Can you give an example of where you would expect different configure options on different cross compile systems to produce the same build as each other?
These options are very important to us for when users report issues, since the issue can quite often be caused by selection of inappropriate configure options at build time.
If different configure options are chosen, then a different executable will be produced, since it will contain different options. My understanding is, therefore, that if different configure options are selected, the build will not be reproducible, in the sense that the binaries will be different.
That is correct
Can you give an example of where you would expect different configure options on different cross compile systems to produce the same build as each other?
That is not what I meant
If you look at the output of keepalived -v in the above post, you will see that there are also paths in the configure options.
These are different for cross compilation.
Here I am mainly concerned with the paths. On our build system there is the user bbworker. Therefore, the paths start with /home/bbworker. But on my developer machine, the paths start with /home/feckert. Despite I have the same build option and feature set, the bin checksum of the builds is different. From my point of view, they should be the same, because they are built with the same toolchain and with the same keepalived version and the same feature set.
Only the build path is diffrent
@pqarmitage I have stumbled across this problem again. How can we continue here now. I have done some research and found on debian the following information for keepalived to make them reproducable https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/keepalived.html. The gcc change is pending since years now!
It seems that other packages also have the problem that they are not reproducible if you include the paths in there executable. As a quick fix, we could remove the information that varies from build machine to build machine. Here it is mainly about the build path.
For keepalived in openwrt the following information by calling keepalived --version on the target system are interesting.
Extract from the configure options that broke the reproducible build:
PKG_CONFIG=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/host/bin/pkg-config
PKG_CONFIG_PATH=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/lib/pkgconfig:/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/share/pkgconfig
PKG_CONFIG_LIBDIR=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/lib/pkgconfig:/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/share/pkgconfig CC=x86_64-openwrt-linux-musl-gcc
CFLAGS=-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/keepalived-2.2.8=keepalived-2.2.8 -Wformat -Werror=format-security -DPIC -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/linux-x86_64/linux-5.15.132
LDFLAGS=-L/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/usr/lib -L/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/lib -fuse-ld=bfd -DPIC -fpic -specs=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/include/hardened-ld-pie.specs -znow -zrelro
CPPFLAGS=-I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/include/fortify -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/include
So could we not remove for now the whole configure options: output, to make keepalived reproducable. Everything else should fit and should be independent of the build system.
I haven't been willing to simply remove the configure_options output, since that information has on occasions been useful in helping to resolve reported issues.
It occurred to me yesterday that the configuration options information could be written to a separate file that keepalived reads at runtime - that way you get a reproducible build for the keepalived binary, and I still get the desired information in the keepalived -v output (and also in system logs when keepalived coredumps).
To enable reproducible builds, and --enable-reproducible-build as a configure option. This will create an additional file keepalived/keepalived.config-opts in the build tree, and make install DESTDIR=... installs the file to $(DESTDIR)/etc/keepalived/keepalived.config-opts. This file MUST be installed in /etc/keepalived, or whatever directory is used for the default configuration file, so your package build scripts may well need updating.
I am sorry this has taken so long to resolve, but it was only yesterday that the idea of implementing it like this occurred to me.
Thank you very much for fixing this issue. On the next release the keepalived could be build reproducible.
I am sorry this has taken so long to resolve, but it was only yesterday that the idea of implementing it like this occurred to me.
I have too thank you :+1: