DietPi icon indicating copy to clipboard operation
DietPi copied to clipboard

build unbound from source

Open z3huti opened this issue 6 months ago • 6 comments

is it possible that you guys can make an option when installing Unbound from dietpi-software we get options to install Unbound from

stable repo testing repo build from source (latest version)

z3huti avatar Jun 03 '25 14:06 z3huti

We do not offer multiple such options which require entirely different installation procedures each. What do you mean with "testing repo"? It is currently installed from the used Debian repo. So on a Bookworm system it is the stable Debian repo, on a Trixie system the testing one. But in both cases it is based on stable Unbound release source code.

I was thinking about finishing this PR to do and provide own Unbound packages based on recent upstream source code: https://github.com/MichaIng/DietPi/pull/7037/files It is currently not implemented correctly into our scripts as is. Would not be hard to finish, but has low priority for me, also as of Debian Trixie release in probably 2 months or so, shipping a pretty recent Unbound already.

Is there anything particular you need from a recent Unbound version? And would using a Debian Trixie DietPi image an option? It provides v1.22.0: https://packages.debian.org/unbound

MichaIng avatar Jun 03 '25 15:06 MichaIng

What do you mean with "testing repo"?

i mean installing Unbound from Debian testing

I was thinking about finishing this PR to do and provide own Unbound packages based on recent upstream source code

that would be awesome!

Is there anything particular you need from a recent Unbound version?

Not really. 1.17 just works, but knowing that this version is from January 2023, and looking at big change logs from newer versions, just want to have the last one available

i was able to update 1.17 to 1.22 from Debian testing repo on a working rpi

and today on VM installed 1.23 from source, just to test how it will go, was not easy for me, did it with AI help =)

And would using a Debian Trixie DietPi image an option?

sound good to me, where i can download one ? maybe arch based one too? =)

imo DietPi is the best distribution for SBC, very easy to use even for beginners, small and fast!

z3huti avatar Jun 03 '25 15:06 z3huti

i mean installing Unbound from Debian testing

That does usually not work, at least not sustainably, unless you fully switch to Debian testing with the whole system. Packages depend on shared libraries, and often specific versions of them, which are only available on the distro version they were compiled for. In some cases it works, but can break any time, when testing gets an upgrade of a shared library. One can be lucky if the Debian package maintainer provides the newer software version via backports suite to older Debian, like it happened on Bullseye, but not for Bookworm. And now, with Trixie release soon, they won't start the hassle of a backport anymore.

sound good to me, where i can download one ? maybe arch based one too? =)

DietPi works only with Debian basis. Here are all images, including those with Debian Trixie: https://dietpi.com/downloads/images/

MichaIng avatar Jun 03 '25 16:06 MichaIng

with Debian Trixie

testing now

systemctl status unbound gives

Jun 03 18:54:16 DietPi (unbound)[477]: unbound.service: Referenced but unset environment variable evaluates to an empty string: DAEMON_OPTS

did some changes to remove this warning, other than that it work ok

z3huti avatar Jun 03 '25 17:06 z3huti

No need to mute that warning. I mean you could declare the variable explicitly as empty string in the environment file, but it is not uncommon that services do not ship one at all, but do still load it if present, and use supported variables, if defined.

MichaIng avatar Jun 03 '25 17:06 MichaIng

I've asked this a couple times in the Discussion forum..

https://github.com/MichaIng/DietPi/discussions/7366

https://github.com/MichaIng/DietPi/discussions/7019

Would be good to have the latest version available with the enhancements etc.

Dynamic5912 avatar Jun 03 '25 21:06 Dynamic5912

if you are gonna provide own newer builds, please make it optional when installing from dietpi-software

btw dietpi on bookworm feels/runs snappier and faster than on trixie

z3huti avatar Jun 10 '25 07:06 z3huti

if you are gonna provide own newer builds, please make it optional when installing from dietpi-software

I would like to avoid making it too complex. Like in some other cases, we would provide it via our APT repository. So our builds would then be installed automatically. It can be always prevented via /etc/apt/preferences.d.

btw dietpi on bookworm feels/runs snappier and faster than on trixie

Which hardware, and which part/software exactly feels slower on Trixie? The DietPi version is not related to the kernel version, at least not on SBCs, so it would be some software component, and hence limited to a particular interface.

MichaIng avatar Jun 10 '25 14:06 MichaIng

Which hardware, and which part/software exactly feels slower on Trixie?

on rpi 3b, startup time 1.5-2 time slower with trixie, dont remember times exactly, something like 3.5 sec and 7 sec on trixie. was only using unbound and pihole, also websites load much faster with bookworm so i am back to dietpi on bookworm with unbound 1.17 and latest pihole

on VM with unbound and pihole Startup finished in 1.109s (kernel) + 3.595s (userspace) = 4.704s - Bookworm Startup finished in 2.389s (kernel) + 4.852s (userspace) = 7.241s - Trixie

z3huti avatar Jun 10 '25 14:06 z3huti

On the Bookworm system, you use the new Linux 6.12 firmware stack already? If so, can you compare

dmesg

timestamps to see at which point they start to differ?

"Startup time" generally is a very vague term. Many things are initiated asynchronously, and there are different stages, bootloader, in case initramfs (not by default used with our RPi images), kernel, init system. If the kernel is the same, the init system should start at the very same point. There are some things which need to be initiated first, like the rootfs mount of course, and other "auto" mounts, which is /boot/firmware on RPi additionally, and some init system core components. Network and e.g. SSH server are brought up before local login prompts, so it can be possible (depending on how fast the network adapter initiates, and then e.g. DHCP server and other software components) to login via SSH earlier than locally. Local console login prompts appear after systemd-user-sessions.service, but other services can start well after login prompts. systemd-analyze shows the time it took for really all services scheduled to be started, which can include Unbound and Pi-hole, while both could hang unlimited without affecting any other part of the host system. What I am saying: it is complex, depends on what you define as a finished startup, what you look at, how you measure etc. The Debian version cannot make the hardware slower of course.

Also assure that you in case use the same overclocking in /boot/config.txt, i.e. that cpu shows the same governor and min/max frequencies. Also the SD card (or boot media) used can affect this, of course.

If hardware, kernel and performance settings are the same, Pi-hole web UI can only be affected by Pi-hole itself. So if it was the same version before and after, any different would be quite surprising: it does not even depends on Debian's PHP packages since Pi-hole v6, but is completely self-contained. Only left explanation could then be high activity or memory usage from other processes slowing down Pi-hole indirectly. Check via htop. Ah, also verify swap space is the same, on the same drive etc, if used at all.

MichaIng avatar Jun 10 '25 14:06 MichaIng

on VM Bookworm is on 6.1.0-37-amd64 Trixie is on 6.12.30-amd64

on RPi 3b Bookworm 6.12.25+rpt-rpi-v8

maybe in few days gonna test Trixie on RPI3b again

Image Image

z3huti avatar Jun 10 '25 16:06 z3huti

x86_64 systems are a different thing, since they use different kernel versions on different Debian versions, which can affect performance, have additional kernel modules loaded etc. Of course one would expect better performance instead of worse, but complexity may raise, or security features raise load/decrease performance. On RPi however, the kernel is exactly the same package in both cases, as you have the new firmware stack on your Bookworm system already.

In htop setup, you can unhide userland and kernel threads, to see where the difference is coming from:

Image

Kernel threads on RPi in both cases will be the same, userland threads might be coming from one of the systemd services, but unlikely has any effect on load or memory usage. Also your screenshots show 2 MiB difference only, which can be coming from various other things, and no load. To find out what is taking longer in boot process, you need to compare dmesg output and in case check which service/target is delayed by what, or which one took longer to switch from "starting" to "started". Quite possible that one of the systemd services at boot became more complex, taking longer to finish.

MichaIng avatar Jun 10 '25 18:06 MichaIng

I compared it on my test VMs where the difference is ~9s Bookworm vs ~10s Trixie:

  • The Trixie system is actually 0.1-0.2 seconds earlier until root partition detection.
  • There it swaps, as somehow detecting the root partition takes 0.25s on Trixie, but less than 0.07s on Bookworm.
  • The difference raises to >0.5s as on Trixie, systemd takes significantly longer to query systemd units.
  • In userland, looks like everything took slightly longer on Trixie.

Bookworm

[    5.602769] sd 0:0:0:0: [sda] 33554432 512-byte logical blocks: (17.2 GB/16.0 GiB)
[    5.610840] sd 0:0:0:0: [sda] Write Protect is off
[    5.616047] sd 0:0:0:0: [sda] Mode Sense: 04 00 10 00
[    5.617766] sd 0:0:0:0: [sda] Incomplete mode parameter data
[    5.624366] sd 0:0:0:0: [sda] Assuming drive cache: write through
[    5.690733]  sda: sda1
[    5.694458] sd 0:0:0:0: [sda] Attached SCSI disk
[    5.727610] EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none.
[    5.751730] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
[    6.095111] systemd[1]: Inserted module 'autofs4'
[    6.174691] systemd[1]: systemd 252.36-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACLORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
...
[    6.232606] systemd[1]: Hostname set to <VM-Bookworm>.
[    6.243165] systemd[1]: Invalid DMI field header.
[    6.649929] systemd[1]: Queued start job for default target graphical.target.
root@VM-Bookworm:~# systemd-analyze blame
1.900s dev-sda1.device
1.864s ifupdown-pre.service
 422ms kmod-static-nodes.service
 418ms dev-hugepages.mount
 418ms dev-mqueue.mount
 415ms sys-kernel-debug.mount
 414ms sys-kernel-tracing.mount
 406ms [email protected]
 387ms modprobe@dm_mod.service
 375ms dietpi-ramlog.service
 374ms modprobe@efi_pstore.service
 352ms [email protected]
 329ms [email protected]
 293ms systemd-udev-trigger.service
 281ms systemd-fsck-root.service
 200ms systemd-modules-load.service
 191ms dietpi-preboot.service
 180ms networking.service
 155ms systemd-journald.service
 155ms systemd-udevd.service
 149ms [email protected]
  93ms sys-fs-fuse-connections.mount
  86ms sys-kernel-config.mount
  69ms systemd-remount-fs.service
  64ms systemd-sysctl.service
  58ms systemd-tmpfiles-setup.service
  49ms systemd-tmpfiles-setup-dev.service
  49ms systemd-update-utmp.service
  45ms systemd-random-seed.service
  44ms tmp.mount
  41ms systemd-sysusers.service
  41ms var-log.mount
  38ms systemd-tmpfiles-clean.service
  32ms systemd-update-utmp-runlevel.service
  24ms systemd-journal-flush.service
  22ms systemd-user-sessions.service

Trixie

[    5.472851] sd 0:0:0:0: [sda] 33554432 512-byte logical blocks: (17.2 GB/16.0 GiB)
[    5.482634] sd 0:0:0:0: [sda] Write Protect is off
[    5.488094] sd 0:0:0:0: [sda] Mode Sense: 04 00 10 00
[    5.489080] sd 0:0:0:0: [sda] Incomplete mode parameter data
[    5.495088] sd 0:0:0:0: [sda] Assuming drive cache: write through
[    5.741674]  sda: sda1
[    5.746877] sd 0:0:0:0: [sda] Attached SCSI disk
[    5.776877] EXT4-fs (sda1): mounted filesystem d2827ac0-ae5f-4239-afa2-8c6c42eac2a7 ro with ordered data mode. Quota mode: none.
[    5.805597] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
[    6.172953] systemd[1]: Inserted module 'autofs4'
[    6.288663] systemd[1]: systemd 257.5-2 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLB +ZSTD +BPF_FRAMEWORK +BTF -XKBCOMMON -UTMP +SYSVINIT +LIBARCHIVE)
...
[    6.364296] systemd[1]: Hostname set to <VM-Trixie>.
[    6.888378] systemd[1]: bpf-restrict-fs: LSM BPF program attached
[    7.294169] systemd[1]: Queued start job for default target graphical.target.
root@VM-Trixie:~# systemd-analyze blame
2.288s dev-sda1.device
2.036s ifupdown-pre.service
 537ms dev-hugepages.mount
 533ms [email protected]
 521ms dev-mqueue.mount
 513ms run-lock.mount
 510ms systemd-fsck-root.service
 506ms sys-kernel-debug.mount
 506ms modprobe@efi_pstore.service
 503ms [email protected]
 500ms sys-kernel-tracing.mount
 498ms kmod-static-nodes.service
 443ms systemd-modules-load.service
 416ms systemd-journald.service
 323ms grub-common.service
 319ms systemd-udev-load-credentials.service
 304ms systemd-udev-trigger.service
 249ms networking.service
 227ms dietpi-ramlog.service
 218ms systemd-tmpfiles-setup.service
 188ms [email protected]
 182ms systemd-udevd.service
 162ms fstrim.service
 158ms systemd-tmpfiles-setup-dev-early.service
 141ms dietpi-preboot.service
 133ms systemd-journal-flush.service
 126ms systemd-remount-fs.service
 120ms var-log.mount
 107ms tmp.mount
 104ms sys-fs-fuse-connections.mount
 104ms sys-kernel-config.mount
 104ms systemd-tmpfiles-clean.service
  98ms systemd-sysctl.service
  67ms systemd-random-seed.service
  65ms systemd-tmpfiles-setup-dev.service
  64ms systemd-user-sessions.service

There are variations on every boot, but a difference remains. systemd definitely became more complex, some changes which caused me a lot of headache when dealing with systems starting in containers for building and testing, with and without QEMU emulation. Looks like this leads to common calls involved in starting up services taking slightly longer, stacking up along the boot procedure. But it should not have an effect on Pi-hole web UI, where systemd is not involved anymore.

MichaIng avatar Jun 10 '25 19:06 MichaIng

did clean trixie install and all running fast and smooth as on bookworm. boot time is just 2 sec slower with trixie on rpi 3b could be that something was wrong with previous trixie install i did a week ago

z3huti avatar Jun 10 '25 20:06 z3huti

Sounds good. Quite realistic that this added systemd boot time on my VM blows up to 2 seconds on a (slower) RPi. Still not great that newer distro versions tend to become larger and more complex as if this was a natural law, but given that newer hardware also becomes faster, it is at least compensated in average. My RPi 2 took 20 seconds and more also on Debian Jessie, IIRC.

MichaIng avatar Jun 10 '25 20:06 MichaIng

Is this possible with Unbound?

Will it be reviewed and implemented at some point?

Dynamic5912 avatar Jun 17 '25 16:06 Dynamic5912

It needs to be implemented differently, will update it after upcoming DietPi release.

MichaIng avatar Jun 17 '25 21:06 MichaIng

still running trixie on rpi3, and just now saw that there is

deb https://archive.raspberrypi.com/debian bookworm main in /etc/apt/sources.list.d/raspi.list next to deb https://archive.raspberrypi.com/debian trixie main

is there a reason that bookworm repo is in trixie system ?

z3huti avatar Jun 24 '25 11:06 z3huti

Yes, the Trixie suite of the RPi repo was/is not complete yet.

MichaIng avatar Jun 24 '25 11:06 MichaIng

I merged a workflow for own builds into the just released DietPi v9.15 beta. Will probably give those packages some more time for testing and in case polishing, before shipping them with our APT repository. For now, if you want to test:

  1. Not necessarily required, but makes sense to test along with some other not strictly related changes around Unbound, update to DietPi v9.15 beta: #7637
  2. Then download and install our package:
    cd /tmp
    wget "https://dietpi.com/downloads/binaries/$G_DISTRO_NAME/unbound_$G_HW_ARCH_NAME.deb"
    apt install ./"unbound_$G_HW_ARCH_NAME.deb"
    

All details here: #7632 At best also use that pull request for discussing details, since there are currently multiple issues and discussions split.

MichaIng avatar Jul 26 '25 18:07 MichaIng

updated to beta and installed(update from 1.22) new Unbound 1.23.1, did some small tests, all is good

z3huti avatar Jul 26 '25 19:07 z3huti

Does it install with the same switches and modules etc. as the current apt version?

_dietpi@dietpi:~$ sudo unbound -V Version 1.17.1

Configure line: --build=aarch64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/aarch64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --with-pythonmodule --with-pyunbound --enable-subnet --enable-dnstap --enable-systemd --with-libnghttp2 --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --disable-rpath --with-pidfile=/run/unbound.pid --with-libevent --enable-tfo-client --with-rootkey-file=/usr/share/dns/root.key --disable-flto --enable-tfo-server Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.0.16 11 Feb 2025

Linked modules: dns64 python subnetcache respip validator iterator TCP Fastopen feature available_

Dynamic5912 avatar Jul 26 '25 20:07 Dynamic5912

Does it install with the same switches and modules etc. as the current apt version?

_dietpi@dietpi:~$ sudo unbound -V Version 1.17.1

Configure line: --build=aarch64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/aarch64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --with-pythonmodule --with-pyunbound --enable-subnet --enable-dnstap --enable-systemd --with-libnghttp2 --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --disable-rpath --with-pidfile=/run/unbound.pid --with-libevent --enable-tfo-client --with-rootkey-file=/usr/share/dns/root.key --disable-flto --enable-tfo-server Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.0.16 11 Feb 2025

Linked modules: dns64 python subnetcache respip validator iterator TCP Fastopen feature available_

sudo unbound -V Version 1.23.1

Configure line: --enable-checking --prefix=/usr --sysconfdir=/etc --localstatedir=/var --runstatedir=/run --without-pyunbound --without-pythonmodule --enable-systemd --enable-subnet --enable-cachedb --enable-dnstap --enable-tfo-client --enable-tfo-server --with-libhiredis --with-libnghttp2 --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --with-libevent --with-pthreads --with-rootkey-file=/usr/share/dns/root.key --disable-rpath Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.5.1 1 Jul 2025 Linked modules: dns64 cachedb subnetcache respip validator iterator TCP Fastopen feature available

BSD licensed, see LICENSE in source package for details. Report bugs to [email protected] or https://github.com/NLnetLabs/unbound/issues

z3huti avatar Jul 26 '25 20:07 z3huti

What happened to your existing config file? Is it overwritten with a new one each time?

I also use unbound-control - no biggie if it's not included anymore

Dynamic5912 avatar Jul 26 '25 20:07 Dynamic5912

What happened to your existing config file? Is it overwritten with a new one each time?

I also use unbound-control - no biggie if it's not included anymore

no, config file is not overwritten, it was just an update of package

z3huti avatar Jul 26 '25 20:07 z3huti

Ok great.

Regarding root-hints not needing monthly updating - mine last had an update in June 2025.. @MichaIng

Dynamic5912 avatar Jul 26 '25 20:07 Dynamic5912

Does it install with the same switches and modules etc. as the current apt version?

All relevant differences are mentioned in the pull request. Otherwise also the splitting off of unbound-anchor and unbound-host done by Debian is done for our package. Hence you can install the two dedicated packages, also python3-unbound from the Debian repo, as well as (which is a dependency of those) libunbound, which is all package-compatible.

What happened to your existing config file? Is it overwritten with a new one each time?

Same as with regular Debian packages, all configs are package conffiles, hence are only updated if they were never touched before. If they were changed (this is checked by dpkg via MD5 checksums), the old config files is left in place, and the new is stored beside it with a .dpkg-dist suffix.

I also use unbound-control - no biggie if it's not included anymore

I think I will re-add the config file Debian maintainers added to their package, to enable the control socket OOTB. Generally, if an additional API is not regularly used, and is not enabled upstream by default, I tend to keep it disabled. In the end it is also an additional attack vector. Unbound defaults however also listens on TCP, while Debian actively changes this to listen to a UNIX socket, least overhead and least vulnerable. So should be fine, and is non-breaking in these regards. I mean otherwise it is only adding one config key. For now, when testing, you will notice the /etc/unbound/unbound.conf.d/remote-control.conf stays in place regardless, since dpkg weirdly adds previous config files to the new package as if they were static files. So I guess it will be only removed when upgrading the package again to a newer version. But any next version will contain it again as config file, so the control socket remains active throughout beta and upgrades.

Regarding root-hints not needing monthly updating - mine last had an update in June 2025..

Due to our cron job yes, but the content did not change in over 2 years: https://github.com/NLnetLabs/unbound/commit/a8739ba The 2nd last change was 9 years ago, and it was an addition, not a changed IP: https://github.com/NLnetLabs/unbound/commit/5692889 Since Debian uses hardcoded Unbound versions for every release, there is in fact a risk that builtin root hints become outdated, i.e. one root server IP, but even despite that, Debian does not configure any external root hints by default, relying on Unbound's builtin list. They do update the root trust anchor, which changes a bit more regularly. But it contains two keys, hence if one expires, the other once can still serve for bootstrapping. Unbound updates the root trust anchors by itself from that point on, which is why you see the /var/lib/unbound/root.key changing regularly (also the content, at least timestamps text within). But if the file does not exist, it can also use builtin data to bootstrap it. Not a good option for Debian's hardcoded Unbound version. But if we update our own package regularly, there really is no need to provide any external data, but we can rely on up-to-date upstream Unbound data instead for both, the root trust anchor as well as root hints for DNSSEC, eliminating possible points of failures and additional sources and procedures you otherwise need to trust.

EDIT: So both Debian config files are now included in our package, leaving unbound-control functional OOTB: https://github.com/MichaIng/DietPi/commit/f544cf1 That "is is" typo hurts my eyes 😄. But it needs to match the Debian config file 100% to replace it. Else we just have another .dpkg-dist.

I couldn't resist 😅: https://salsa.debian.org/dns-team/unbound/-/merge_requests/23

MichaIng avatar Jul 26 '25 21:07 MichaIng

I am closing it here, let's keep discussion in #7632.

MichaIng avatar Jul 28 '25 23:07 MichaIng