openzfs
zfs-2.2.0 building native-deb-utils target fails early on
I don't file many bug reports. I did my best to fill out the template and provide useful information. I have no fixes for the issue so far. Thank you.
There was a lot more dh_missing errors, however I hit the max character limit in a bug report, which is fair.
System information
Distribution Name | Ubuntu Distribution Version | 22.04.03 Kernel Version | 6.2.0-32-generic Architecture | AMD64 OpenZFS Version | 2.2.0
zfs-2.1.12-1 zfs-kmod-2.1.12-1 6.2.0-32-generic
Describe the problem you're observing
building native-deb or native-deb-utils fails. I thought this would work per: https://openzfs.github.io/openzfs-docs/Developer%20Resources/Building%20ZFS.html#build-options Support for native Debian packaging will be available starting from openzfs-2.2 release.
Describe how to reproduce the problem
extract fresh 2.2.0 source tree
./configure --enable-systemd
make native-deb
cp -r contrib/debian debian; chmod +x debian/rules;
cp contrib/debian/control debian/control; \
dpkg-buildpackage -b -rfakeroot -us -uc;
cp: cannot stat 'contrib/debian/control': No such file or directory
dpkg-buildpackage: error: cannot open file debian/changelog: No such file or directory
make: *** [Makefile:14317: native-deb-utils] Error 255
If I copy over contrib/debian/control* and changelog from git version 2.2.0 into the 2.2.0 release directory, it builds a bit longer, but then seems to fail when it gets to
make[2]: Entering directory '/home/user/test/zfs-2.2.0'
dh_missing --fail-missing
dh_missing: warning: etc/default/zfs exists in debian/tmp but is not installed to anywhere
dh_missing: warning: etc/init.d/zfs-import exists in debian/tmp but is not installed to anywhere
dh_missing: warning: lib/udev/rules.d/90-zfs.rules exists in debian/tmp but is not installed to anywhere
dh_missing: warning: lib/udev/vdev_id exists in debian/tmp but is not installed to anywhere
dh_missing: warning: lib/udev/zvol_id exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/lib/decompress/zstd_decompress_block.c exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/lib/decompress/zstd_decompress_block.h exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/lib/decompress/zstd_decompress_internal.h exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/lib/zstd.h exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/zfs_zstd.c exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/zstd-in.c exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/module/zstd/zstd_sparc.c exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/scripts/dkms.postbuild exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/scripts/enum-extract.pl exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/zfs.release.in exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/src/zfs-2.2.0/zfs_config.h.in exists in debian/tmp but is not installed to anywhere
The following debhelper tools have reported what they installed (with files per package)
* dh_install: openzfs-libnvpair3 (0), openzfs-libpam-zfs (0), openzfs-libuutil3 (0), openzfs-libzfs-dev (0), openzfs-libzfs4 (0), openzfs-libzfsbootenv1 (0), openzfs-libzpool5 (0), openzfs-python3-pyzfs (0), openzfs-pyzfs-doc (0), openzfs-zfs-dkms (0), openzfs-zfs-dracut (0), openzfs-zfs-initramfs (0), openzfs-zfs-test (0), openzfs-zfs-zed (0), openzfs-zfsutils (0)
* dh_installdocs: openzfs-libnvpair3 (0), openzfs-libpam-zfs (0), openzfs-libuutil3 (0), openzfs-libzfs-dev (0), openzfs-libzfs4 (0), openzfs-libzfsbootenv1 (0), openzfs-libzpool5 (0), openzfs-python3-pyzfs (0), openzfs-pyzfs-doc (0), openzfs-zfs-dkms (0), openzfs-zfs-dracut (0), openzfs-zfs-initramfs (0), openzfs-zfs-test (0), openzfs-zfs-zed (0), openzfs-zfsutils (0)
If the missing files are installed by another tool, please file a bug against it.
When filing the report, if the tool is not part of debhelper itself, please reference the
"Logging helpers and dh_missing" section from the "PROGRAMMING" guide for debhelper (10.6.3+).
(in the debhelper package: /usr/share/doc/debhelper/PROGRAMMING.gz)
Be sure to test with dpkg-buildpackage -A/-B as the results may vary when only a subset is built
If the omission is intentional or no other helper can take care of this consider adding the
paths to debian/not-installed.
Remember to be careful with paths containing "x86_64-linux-gnu", where you might need to
use a wildcard or (assuming compat 13+) e.g. ${DEB_HOST_MULTIARCH} in debian/not-installed
to ensure it works on all architectures (see #961104).
dh_missing: error: missing files, aborting
make[2]: *** [debian/rules:160: override_dh_missing] Error 255
make[2]: Leaving directory '/home/user/test/zfs-2.2.0'
make[1]: *** [debian/rules:28: binary] Error 2
make[1]: Leaving directory '/home/user/test/zfs-2.2.0'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make: *** [Makefile:14317: native-deb-utils] Error 2
Also make native-deb-kmod fails with
/bin/sh: 1: cannot open debian/control.modules.in: No such file
make[1]: *** [debian/rules:185: override_dh_prep-deb-files] Error 2
make: *** [Makefile:14320: native-deb-kmod] Error 2
If I copy over that file from the 2.2.0 tagged git source, I get
sh scripts/make_gitrev.sh; \
fakeroot debian/rules override_dh_binary-modules;
make[1]: Entering directory '/home/user/test/zfs-2.2.0'
for templ in ; do \
sed -e 's/##KVERS##/6.2.0-32-generic/g ; s/#KVERS#/6.2.0-32-generic/g ; s/_KVERS_/6.2.0-32-generic/g ; s/##KDREV##//g ; s/#KDREV#//g ; s/_KDREV_//g ; s/_ARCH_/amd64/' \
< $templ > `echo $templ | sed -e 's/_KVERS_/6.2.0-32-generic/g ; s/_ARCH_/amd64/g ; s/\.in$//'` ; \
done
sed -e 's/##KVERS##/6.2.0-32-generic/g ; s/#KVERS#/6.2.0-32-generic/g ; s/_KVERS_/6.2.0-32-generic/g ; s/##KDREV##//g ; s/#KDREV#//g ; s/_KDREV_//g ; s/_ARCH_/amd64/g' \
< debian/control.modules.in > debian/control
dh override_dh_configure_modules --with autoreconf,dkms,python3,sphinxdoc
dh_testdir
dh_testroot
dh_prep
/usr/bin/make -j32 -C /home/user/test/zfs-2.2.0/module modules
make[2]: Entering directory '/home/user/test/zfs-2.2.0/module'
mkdir -p os/linux/spl/
mkdir -p avl/ icp/ icp/algs/aes/ icp/algs/blake3/ icp/algs/edonr/ icp/algs/modes/ icp/algs/sha2/ icp/algs/skein/ icp/api/ icp/asm-aarch64/blake3/ icp/asm-aarch64/sha2/ icp/asm-arm/sha2/ icp/asm-ppc64/blake3/ icp/asm-ppc64/sha2/ icp/asm-x86_64/aes/ icp/asm-x86_64/blake3/ icp/asm-x86_64/modes/ icp/asm-x86_64/sha2/ icp/core/ icp/io/ icp/spi/ lua/ lua/setjmp/ nvpair/ os/linux/zfs/ unicode/ zcommon/ zfs/ zstd/ zstd/lib/common/ zstd/lib/compress/ zstd/lib/decompress/
/usr/bin/make -C \
\
M="$PWD" CONFIG_ZFS=m modules
make[3]: Entering directory '/home/user/test/zfs-2.2.0/module'
make[3]: *** M=/home/user/test/zfs-2.2.0/module: No such file or directory. Stop.
make[3]: Leaving directory '/home/user/test/zfs-2.2.0/module'
make[2]: *** [Makefile:56: modules-Linux] Error 2
make[2]: Leaving directory '/home/user/test/zfs-2.2.0/module'
make[1]: *** [debian/rules:203: override_dh_binary-modules] Error 2
make[1]: Leaving directory '/home/user/test/zfs-2.2.0'
make: *** [Makefile:14320: native-deb-kmod] Error 2
If I copy over that file from the 2.2.0 tagged git source, I get
Since you already checked out zfs-2.2-release branch - just build from there. It works.
# ./autogen.sh
# ./configure
# make native-deb
The release tar is broken, yeah.
It looks like the zfs-2.2.0.tar.gz on the release page only contains rules.in under contrib/debian, other contents are missing. However, the Source code (zip) and Source code (tar.gz) do seem to have all the contents.
I was able to successfully build the Debian packages with Source code (zip) on Debian 12, as mentioned above:
# sh autogen.sh
# ./configure
# make native-deb
- release tar is missing files
git cloneinstructions should probably also recommend checking out a specific tag (e.g.zfs-2.2-release)- two build dependencies are not listed: libcurl4-openssl-dev libpam0g-dev
- the instructions specify to
make native-deb-utilsbut @blind-oracle is givingmake native-deb, what's the difference and which one is correct? - I can't install the openzfs-zfs-dkms package on ubuntu server 22.04.3 because it conflicts with the zfs-dkms provided by the kernel:
$ sudo apt install ./openzfs-zfs-zed_2.2.0-0_amd64.deb ./openzfs-zfsutils_2.2.0-0_amd64.deb ./openzfs-zfs-dkms_2.2.0-0_all.deb ./openzfs-libzpool5_2.2.0-0_amd64.deb ./openzfs-libzfs4_2.2.0-0_amd64.deb ./openzfs-libnvpair3_2.2.0-0_amd64.deb ./openzfs-libuutil3_2.2.0-0_amd64.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'openzfs-zfs-zed' instead of './openzfs-zfs-zed_2.2.0-0_amd64.deb'
Note, selecting 'openzfs-zfsutils' instead of './openzfs-zfsutils_2.2.0-0_amd64.deb'
Note, selecting 'openzfs-zfs-dkms' instead of './openzfs-zfs-dkms_2.2.0-0_all.deb'
Note, selecting 'openzfs-libzpool5' instead of './openzfs-libzpool5_2.2.0-0_amd64.deb'
Note, selecting 'openzfs-libzfs4' instead of './openzfs-libzfs4_2.2.0-0_amd64.deb'
Note, selecting 'openzfs-libnvpair3' instead of './openzfs-libnvpair3_2.2.0-0_amd64.deb'
Note, selecting 'openzfs-libuutil3' instead of './openzfs-libuutil3_2.2.0-0_amd64.deb'
The following packages were automatically installed and are no longer required:
amd64-microcode libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 libusbmuxd6 linux-headers-generic-hwe-22.04 thermald upower usbmuxd
Use 'sudo apt autoremove' to remove them.
Suggested packages:
debhelper nfs-kernel-server openzfs-zfs-initramfs | openzfs-zfs-dracut
The following packages will be REMOVED:
linux-generic linux-generic-hwe-20.04 linux-generic-hwe-22.04 linux-image-5.15.0-86-generic linux-image-6.2.0-33-generic linux-image-6.2.0-34-generic linux-image-generic linux-image-generic-hwe-22.04
linux-modules-5.15.0-86-generic linux-modules-extra-5.15.0-86-generic
The following NEW packages will be installed:
openzfs-libnvpair3 openzfs-libuutil3 openzfs-libzfs4 openzfs-libzpool5 openzfs-zfs-dkms openzfs-zfs-zed openzfs-zfsutils
0 upgraded, 7 newly installed, 10 to remove and 0 not upgraded.
Need to get 0 B/4,851 kB of archives.
After this operation, 480 MB disk space will be freed.
@mwpastore Yes, these native deb packages are in some weird state, probably gonna have to postpone upgrade to 2.2 until those conflicts are fixed...
There are three targets for native packaging, make native-deb, make native-deb-utils and make native-deb-kmod:
make native-deb-utils: Builds the userland utilities and the DKMS package. If kernel modules are to be installed via DKMS package, this target should be sufficient.
make native-deb-kmod: Builds the kernel modules and package the binaries.
make native-deb: Runs native-deb-utils and native-deb-kmod. All userland packages, DKMS package and kernel module package should be built.
There two ways to build the Debian packages. RPM converted packages from Alien can be built by using make deb, and native Debian packages, targets for which are mentioned above.
The conflicts were added intentionally with RPM converted Debian packages and the downstream Debian packages. This was done to make sure, the local native Debian packages cannot co-install with other Debian packages maintained by downstream distributions and RPM converted packages.
To install native Debian packages, please remove the previously installed packages and then install the native Debian packages. Alternatively, RPM converted Debian packages can be installed to upgrade the previously installed ones.
To install native Debian packages, please remove the previously installed packages and then install the native Debian packages.
Sadly it does not work like this, at least in Ubuntu - zfs-dkms is provided by the Linux kernel packages and apt refuses to install openzfs-zfs-dkms. Removing kernel, is, obviously a bad idea... :)
Which is actually strange - why kernel should provide DKMS which is purposely created for out-of-kernel modules.
# dpkg -i openzfs-zfs-dkms_2.2.0-0_all.deb
dpkg: regarding openzfs-zfs-dkms_2.2.0-0_all.deb containing openzfs-zfs-dkms:
openzfs-zfs-dkms conflicts with zfs-dkms
linux-image-6.2.0-34-generic provides zfs-dkms and is present and installed.
linux-image-6.2.0-33-generic provides zfs-dkms and is present and installed.
dpkg: error processing archive openzfs-zfs-dkms_2.2.0-0_all.deb (--install):
conflicting packages - not installing openzfs-zfs-dkms
Errors were encountered while processing:
openzfs-zfs-dkms_2.2.0-0_all.deb
@AllKind No it does not. But when you try to install openzfs-zfs-dkms then it would:
# apt install ./openzfs-zfs-dkms_2.2.0-0_all.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'openzfs-zfs-dkms' instead of './openzfs-zfs-dkms_2.2.0-0_all.deb'
The following packages were automatically installed and are no longer required:
amd64-microcode intel-microcode iucode-tool libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 libusbmuxd6 linux-headers-generic-hwe-22.04 thermald upower usbmuxd
Use 'apt autoremove' to remove them.
Recommended packages:
openzfs-zfs-zed openzfs-zfsutils
The following packages will be REMOVED:
linux-generic-hwe-22.04 linux-image-6.2.0-33-generic linux-image-6.2.0-34-generic linux-image-generic-hwe-22.04
The following NEW packages will be installed:
openzfs-zfs-dkms
0 upgraded, 1 newly installed, 4 to remove and 0 not upgraded.
Need to get 0 B/2557 kB of archives.
After this operation, 7845 kB disk space will be freed.
So it's like usaleem-ix said. You have to remove the old package first. Not saying I have any idea why apt wants to uninstall the kernel in the first place. Which sounds like something is wrong.
So it's like usaleem-ix said. You have to remove the old package first.
No, no old packages are installed, I've purged all 2.1.13 stuff. It just wants to remove the kernel, for sure it's wrong.
dpkg -i openzfs-zfs-dkms_2.2.0-0_all.deb dpkg: regarding openzfs-zfs-dkms_2.2.0-0_all.deb containing openzfs-zfs-dkms: openzfs-zfs-dkms conflicts with zfs-dkms
It just looked like you still have zfs-dkms installed.
@AllKind zfs-dkms is some virtual thing that the kernel provides, it just happen to have the same name as zfs-2.1.x package and somehow they don't conflict and can be co-installed. But not in 2.2.x
Can be seen here in the Provides section: https://ubuntu.pkgs.org/22.04/ubuntu-updates-main-arm64/linux-image-unsigned-6.2.0-33-generic_6.2.0-33.33~22.04.1_arm64.deb.html
@blind-oracle I don't think that's true. Provides does not mean the package must be virtual. It can also be a real package name.
searching for all virtual packages and grepping for zfs:
aptitude search "~v" |grep zfs v zfs-modules - v zfsutils -
apt show -a zfs-dkms will show it's a real package and the install status.
Yeah, the conflicts with the provides is probably not going to work. Hm.
I'm not sure what the right way to resolve this is, tbh - Ubuntu's kernel is probably correct to say it provides zfs-dkms, in some sense. Really, it'd be nice if we could have a provides: zfs-modules and then zfs-dkms also provides that and we just have the Conflicts on zfs-dkms, not zfs-modules, but, I don't think that's necessarily feasible to get everyone to do Right Now.
The simplest way might just be to drop the conflicts for zfs-dkms for now and try to get zfs-dkms and Ubuntu's kernel tree to have a different provides so that we can conflict with zfs-dkms the package later, not the virtual thing it's providing. But I'm not an expert in best practices on this.
@AllKind I don't get what you mean. I don't have a real zfs-dkms package installed, but there's still a conflict that makes apt remove the kernel.
@rincebrain Well, I would be happy with how it was working in 2.1.x and before (with alien + rpm conversion). Probably it still works now, but having native debs are probably nicer. There were no conflicts, everything worked like a charm.
Update: I guess the reason is that 2.2 packages have this in the deb definition. So apt tries to remove anything that provides those.
Replaces: spl-dkms, zfs-dkms
I hadn't realized it now forces you to build the native deb packages either way in 2.2. How...well thought out, in literally no way.
@rincebrain maybe the old way works (thru alien), I haven't yet tried. But I wanted to have the native ones if they were announced...
@blind-oracle I was just trying to say, that it's not a virtual package. But I think I just misunderstood, what you were saying with "virtual thing".
I didn't know, that Provides could produce a Conflict. That seems to be the problem.
@rincebrain
What about naming the package names the old way. Without the openzfs- prefix. For me it sounds redundant (and kinda silly). And then just remove zfs-dkms from the Replaces definition?
Wouldn't that make things work like they used to, without the need to change Ubuntus definitions?
@rincebrain maybe the old way works (thru alien), I haven't yet tried. But I wanted to have the native ones if they were announced...
And then you could easily run into https://github.com/openzfs/zfs/issues/15253 which is likely the same as https://github.com/openzfs/zfs/issues/15328. That with the broken tar ball of the 2.2.0 release... Quite a mess installing ZFS right now :-O
The entire point of the prefix was to ensure you couldn't end up with a mix of the two package sets or have to worry about the versioning causing weird hard to debug problems.
You could just drop the Replaces, but then you could wind up with both module packages installed.
You could, I imagine, try having something janky like a post-install step that checks if zfs-dkms is virtual and if not, removes it, if you still wanted to try enforcing that.
Hm, are you saying apt does not recognize version 2.2 is higher than 2.1? And even then, isn't that just postponing the same thing just with different names?
I'm saying that this prefix avoids you winding up with Ubuntu's patched libzfs5-2.2.0 and OpenZFS's vanilla userland, for example.
An additional remark is that, if Ubuntu 24.04, for example, shipped 2.2.0, and you wanted to install OpenZFS's 2.2.0, and hypothetically the package names were the same, apt treats 2.2.0 as less than 2.2.0ubuntu1 for version numbering purposes, I believe, so it would always want to upgrade to the stock packages unless you put an explicit version freeze in.
I can confirm that after removing Conflicts: zfs-dkms (Replaces does not play a role) from contrib/debian/control the dkms deb installs fine, though it shows some strange empty dialog window with a single OK button and some warnings in console then:
debconf: Unknown template field '_description', in stanza #1 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates
debconf: Unknown template field '_description', in stanza #2 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates
debconf: Unknown template field '_description', in stanza #3 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates
I am perplexed as to why the Ubuntu kernel image package provides zfs-dkms and spl-dkms. I can understand why it provides zfs-modules and spl-modules—and it makes sense that those would conflict with modules installed by another package. But it kind of defeats the point of a DKMS package to say that the kernel image package provides it.
Also, why is the provides set on the package containing vmlinux (linux-image-6.2.0-34-generic) and not the metapackage (linux-generic-hwe-22.04) or modules package (linux-modules-6.2.0-34-generic)? Even the headers package (linux-headers-6.2.0-34-generic) would make more sense (to me, but I am naive).
Anyway, I'm sure the good folks over at Ubuntu have sound reasons for having done this—probably to prevent someone shooting themselves in the foot—but it's quite frustrating.
I suppose these new native debs created by OpenZFS have the Conflicts directive in the control file to ensure it doesn't get installed alongside an older zfs-dkms package (i.e. one not provided by the Ubuntu kernel). I wonder if there's a way to write the Conflicts directive so that it excludes such packages without conflicting with the Ubuntu kernel?
EDIT: One other note. I edited the control file in the openzfs-zfs-dkms package I built and removed the Conflicts directive, per @blind-oracle, and it installed fine. However, it didn't automatically uninstall the openzfs-zfs-modules package I installed yesterday. Surely these two things should be mutually exclusive? Maybe the Conflicts was being used for this too?
I can confirm that after removing
Conflicts: zfs-dkms(Replacesdoes not play a role) fromcontrib/debian/controlthe dkms deb installs fine, though it shows some strange empty dialog window with a single OK button and some warnings in console then:debconf: Unknown template field '_description', in stanza #1 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates debconf: Unknown template field '_description', in stanza #2 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates debconf: Unknown template field '_description', in stanza #3 of /var/lib/dpkg/info/openzfs-zfs-dkms.templates
I can confirm the same, https://github.com/EchterAgo/zfs/commit/8553e34f86198590fefce1127b9e7632e68bde21
@mwpastore Yeah I agree that the kernel should not provide anything DKMS related since it defeats DKMS's purpose. Maybe we should file a bug to Ubuntu, but it will take a long time to be acted upon for sure. So I guess we need to decide smth on OpenZFS side for now...
I don't...think that #15503 fixed the original bug reported here, Brian, which was that the 2.2 release tarball is broken by missing key files.
@behlendorf Unless the tarball for 2.2.0 has been fixed the issue remains. That's why I linked the issue comment in my PR.
2.2.1 tar.gz still fails with same
cp: cannot stat 'contrib/debian/control': No such file or directory
~If using git with 2.2.1 tag then it fails later with some cryptic~
Disregard that, after re-cloning the repo it built fine.
@tonyhutter Could you take a look? Why are the 2.2.x release tarballs broken? Is it something that is automated by github, or???