kubler
kubler copied to clipboard
edannenberg
ld.so.cache not updating properly for emerge-root
Hi All,
I've been trying to build an image with the following build.sh (superfluous comments removed):
#
# Kubler phase 1 config, pick installed packages and/or customize the build
#
_packages="
app-crypt/libb2
app-dicts/myspell-en
dev-build/meson
dev-libs/openssl
dev-python/sphinx
dev-python/sphinx-rtd-theme
dev-python/sphinxcontrib-spelling
sys-devel/binutils
"
BOB_INSTALL_BASELAYOUT=true
configure_builder()
{
:
}
configure_rootfs_build()
{
update_use 'app-dicts/myspell-en' '+l10n::en-AU'
update_use 'sys-apps/util-linux' '-su'
:
}
finish_rootfs_build()
{
:
}
Unfortunately, I run into issues where python is unable to find libb2 (blake2{b,s}) when loading hashlib:
ERROR:root:code for hash blake2b was not found.
Traceback (most recent call last):
File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
globals()[__func_name] = __get_hash(__func_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
return __get_builtin_constructor(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2b
ERROR:root:code for hash blake2s was not found.
Traceback (most recent call last):
File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
globals()[__func_name] = __get_hash(__func_name)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
return __get_builtin_constructor(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
usage: sphinx-build [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
From within the container I see
306d835e5dc8 / # ldd /usr/lib64/libb2.so
linux-vdso.so.1 (0x00007ffe94bd9000)
libgomp.so.1 => not found
libc.so.6 => /lib64/libc.so.6 (0x00007d1a249b9000)
/lib64/ld-linux-x86-64.so.2 (0x00007d1a24ba0000)
which is odd as libgomp definitely exists:
306d835e5dc8 / # LD_PRELOAD=/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1 python
Python 3.11.8 (main, Mar 13 2024, 06:07:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>>
and is in /etc/ld.so.conf:
8835fb8f3e6b / # cat /etc/ld.so.conf
# contents of /etc/env.d directory.
include ld.so.conf.d/*.conf
# ld.so.conf autogenerated by env-update; make all changes to
/lib
/lib64
/usr/lib
/usr/lib64
/usr/lib/gcc/x86_64-pc-linux-gnu/13
/usr/local/lib
/usr/local/lib64
but not in the cache:
8835fb8f3e6b / # strings /etc/ld.so.cache
glibc-ld.so.cache1.1.
/usr/lib64/libresolv.so
/usr/lib64/libcrypt.so
/lib64/libskarnet.so
/usr/lib64/libnss_compat.so
/lib64/libs6auto.so
/usr/lib64/libanl.so
/usr/lib64/libc_malloc_debug.so
/usr/lib64/libunistring.so
/lib64/libexecline.so
/lib64/libpcprofile.so
/usr/lib64/libBrokenLocale.so
/lib64/libmemusage.so
/usr/lib64/libnss_hesiod.so
/usr/lib64/libmvec.so
/usr/lib64/libnss_db.so
/usr/lib64/libthread_db.so
/lib64/libs6.so
/usr/lib64/libidn2.so
/lib64/libexecline.so.2.9
/lib64/libm.so.6
/lib64/libc.so.6
/usr/lib64/libunistring.so.5
/lib64/libskarnet.so.2.14
/lib64/libs6auto.so.2.12
/lib64/libs6.so.2.12
/lib64/libresolv.so.2
/usr/lib64/libcrypt.so.2
/lib64/libnss_compat.so.2
/lib64/libnss_dns.so.2
/lib64/libnss_files.so.2
/lib64/libdl.so.2
/lib64/libnss_hesiod.so.2
/lib64/libnss_db.so.2
/lib64/ld-linux-x86-64.so.2
/lib64/librt.so.1
/lib64/libcrypt.so.1
/lib64/libnsl.so.1
/lib64/libanl.so.1
/lib64/libutil.so.1
/lib64/libBrokenLocale.so.1
/lib64/libmvec.so.1
/lib64/libthread_db.so.1
/lib64/libc_malloc_debug.so.0
/lib64/libpthread.so.0
/usr/lib64/libidn2.so.0
ldconfig (Gentoo 2.38-r10 (patchset 10)) stable release version 2.38
Regenerating the cache inside the container resolves the issue, but I shouldn't need to do that:
docker run -it esphpc/sphinx-docs-builder /bin/bash
2ab3065ffba0 / # cat /etc/ld.so.conf.d/05gcc-x86_64-pc-linux-gnu.conf
/usr/lib/gcc/x86_64-pc-linux-gnu/13
2ab3065ffba0 / # ldconfig ; python
Python 3.11.8 (main, Mar 13 2024, 06:07:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>>
I'm also not sure where the cache is being generated, as we remove the cache from emerge-root, maybe that can shed some light on the situation:
https://github.com/edannenberg/kubler/blob/3aa97bb39309840db0e61ece7558e6d6e7a8caec/engine/docker/bob-core/build-root.sh#L708-L709
Any insight on what's happening? Portage's env-update should take care of this and it's ROOT aware.
In the short term I will try to generate the cache as a postinst step before a rootfs tarball is taken.
Following along with that, adding ldconfig -X -r "/emerge-root" --verbose to finish_rootfs_build detects the library:
+ ldconfig -X -r /emerge-root --verbose
ldconfig: Path `/lib' given more than once
(from <builtin>:0 and /etc/ld.so.conf:4)
ldconfig: Path `/lib64' given more than once
(from <builtin>:0 and /etc/ld.so.conf:5)
ldconfig: Can't stat /libx32: No such file or directory
ldconfig: Path `/usr/lib' given more than once
(from <builtin>:0 and /etc/ld.so.conf:6)
ldconfig: Path `/usr/lib64' given more than once
(from <builtin>:0 and /etc/ld.so.conf:7)
ldconfig: Can't stat /usr/libx32: No such file or directory
/lib: (from /etc/ld.so.conf:4)
/lib64: (from /etc/ld.so.conf:5)
/usr/lib: (from /etc/ld.so.conf:6)
/usr/lib64: (from /etc/ld.so.conf:7)
libenchant-2.so.2 -> libenchant-2.so.2.5.0
libgio-2.0.so.0 -> libgio-2.0.so.0.7800.3
libglib-2.0.so.0 -> libglib-2.0.so.0.7800.3
libgmodule-2.0.so.0 -> libgmodule-2.0.so.0.7800.3
libgobject-2.0.so.0 -> libgobject-2.0.so.0.7800.3
libgthread-2.0.so.0 -> libgthread-2.0.so.0.7800.3
libxml2.so.2 -> libxml2.so.2.12.5
libpython3.11.so.1.0 -> libpython3.11.so.1.0
libb2.so.1 -> libb2.so.1.0.4
libmpc.so.3 -> libmpc.so.3.3.1
libmpfr.so.6 -> libmpfr.so.6.2.1
libblkid.so.1 -> libblkid.so.1.1.0
libfdisk.so.1 -> libfdisk.so.1.1.0
libmount.so.1 -> libmount.so.1.1.0
libsmartcols.so.1 -> libsmartcols.so.1.1.0
libuuid.so.1 -> libuuid.so.1.3.0
libasm.so.1 -> libasm-0.190.so
libdw.so.1 -> libdw-0.190.so
libelf.so.1 -> libelf-0.190.so
libpkgconf.so.4 -> libpkgconf.so.4.0.0
libhunspell-1.7.so.0 -> libhunspell.so
libsqlite3.so.0 -> libsqlite3.so.0.8.6
libexpat.so.1 -> libexpat.so.1.8.10
libexpatw.so.1 -> libexpatw.so.1.8.10
libffi.so.8 -> libffi.so.8.1.2
libpcre2-16.so.0 -> libpcre2-16.so.0.11.2
libpcre2-32.so.0 -> libpcre2-32.so.0.11.2
libpcre2-8.so.0 -> libpcre2-8.so.0.11.2
libpcre2-posix.so.3 -> libpcre2-posix.so.3.0.4
libgmp.so.10 -> libgmp.so.10.5.0
libgmpxx.so.4 -> libgmpxx.so.4.7.0
/usr/lib/gcc/x86_64-pc-linux-gnu/13: (from /etc/ld.so.conf:8)
libasan.so.8 -> libasan.so.8.0.0
libatomic.so.1 -> libatomic.so.1.2.0
libcc1.so.0 -> libcc1.so.0.0.0
libgcc_s.so.1 -> libgcc_s.so.1
libgfortran.so.5 -> libgfortran.so.5.0.0
libgomp.so.1 -> libgomp.so.1.0.0
libhwasan.so.0 -> libhwasan.so.0.0.0
libitm.so.1 -> libitm.so.1.0.0
liblsan.so.0 -> liblsan.so.0.0.0
libquadmath.so.0 -> libquadmath.so.0.0.0
libstdc++.so.6 -> libstdc++.so.6.0.32
libtsan.so.2 -> libtsan.so.2.0.0
libubsan.so.1 -> libubsan.so.1.0.0
/usr/local/lib: (from /etc/ld.so.conf:9)
/usr/local/lib64: (from /etc/ld.so.conf:10)
but the ld.so.cache that ends up in the rootfs tarball does not match this. Are we unpacking a cached copy somewhere? Edit: we have to be, because we rming it after running finish_rootfs_build as per the above post.
Moving my workaround to the dockerfile template...