zfs
zfs copied to clipboard
openzfs
Version update for Kernel 6.7 compatibility
Since the kernel version 6.7 has recently been released and Fedora will probably update to this version soon, I wanted to ask if you could release a small update to reach 6.7 compatibility.
As far as i see the needed changes are already merged? #15681
Arch has already released linux 6.7 https://archlinux.org/packages/core/x86_64/linux/
Any ideas if something is being done for this? Typically, this support has been released way before the kernel releases. We are at 6.7.2 already.
Support for 6.7 is already available on master (#15681) and will be in 2.2.3 (#15694).
Thanks for the update @robn! Is there a timeline for official 2.2.3 release and what's pending?
I wonder why tagging 2.2.3 is taking so long when 2.2.1 was tagged a month after 2.2.0 on Nov 13th and 2.2.2 was tagged 2 weeks after 2.2.2 on Nov 28. Its been more than 2 months since 2.2.2.
Is there a showstopper blocking issue for 2.2.3? I thought 2.2.x is now a patchset series which would see very minimal changes and not break much, while 2.3 will take the new features etc. Is this not correct interpretation of release versions?
A point release has certain expectations of stability and it takes time to test and polish. 2.2.3 is also perhaps a little bigger than a normal point release, as 2.2.1 and 2.2.2 were both a little unusual in that they included rush fixes for critical issues. So we really want to take the time to feel confident with it.
The best thing you can do to help is try the staging branch (#15836) and report back on how it goes.
@robn Makes sense. But is there a current blockers list for 2.2.3? I think we should have a time line for release. We shouldn't wait to accumulate fixes beyond a cut off.
I don't have a problem trying out 2.2.3-staging. But I would like to stick with gentoo ebuilds instead of doing point in time git based builds and manually copying the modules. Does anybody have a 2.2.3-staging ebuild that they are using?
@devsk 2.2.3-staging is in nixpkgs-unstable as zfsUnstable. You could try building your kernel + zfs with Nix instead for now if it’s a hassle to do it with Gentoo in the meantime. I have an overlay you can borrow to compile the kernel with gcc for your specific architecture if trying to do the -march=native Gentoo route.
ZFS being my main filesystem for the last 15 years, I have deepest respect for the outstanding effort the developers put into ensuring maximum stability and reliability. In fact, the recent problems with block cloning (and the adventures I had with unreliable RAM a few months ago) only made me appreciate even more how robust ZFS is—which is arguably the single most important characteristic of a filesystem.
Maintaining compatibility with the Linux kernel is also not always the easiest of tasks for different reasons. And this might be somewhat of an understatement.
However, because more streamlined releases would also benefit many users (granted, in enterprise use cases most of us typically use LTS kernels, so these issues usually do not affect us) I've been thinking would it be too cumbersome to have releases with only the changes required for kernel compatibility once a new Linux kernel is released—and leave the rest of the accumulated changes in "master" for the next release? <-- TL;DR
Depending on the status of "master" I guess it might require sometimes significant backporting effort of the kernel-related changes to the latest stable release—which I suppose is the main reason why this isn't done. And it will mean yet another thing to monitor in terms of integration testing.
But there are probably benefits as well—fewer changes in a release usually leads to more easily debuggable problems (or additional noise, of course, because of the additional work to port the changes).
I recognize the intricacies of integrating such a process within the existing development workflow and am curious about the team's perspective on this idea. Needless to say, I understand my suggestion might be oversimplifying these complexities, and—in open-source projects in particular—resources might be rather scarce and thus must be managed very carefully. Even if this means sacrificing otherwise desirable features.
Perhaps there could also be different, more efficient approaches to achieve the same goal of more streamlined kernel-compatibility releases?
Same here @kerberizer. I started using ZFS as zfs-fuse long time ago and have been one of the earliest adopters of the kernel module from its very birth. Brian used to be the lone warrior at the time!
ZFS has saved me several times! So, I am an extremely thankful user!
But is there a current blockers list for 2.2.3?
@devsk I think we'd like to get https://github.com/openzfs/zfs/issues/15842 (block cloning fix) in before releasing . Once it's merged into master we'll pull it into 2.2.3.
We shouldn't wait to accumulate fixes beyond a cut off.
We put out point releases as needed rather than following a strict calendar schedule. Typically a new release is driven by a need for new kernel compatibility fixes, or a pile-up of patches in -staging. I would say our typical cadence is every 3-4 months for point releases, and once a year for major releases.
would it be too cumbersome to have releases with only the changes required for kernel compatibility once a new Linux kernel is released—and leave the rest of the accumulated changes in "master" for the next release
I recognize the intricacies of integrating such a process within the existing development workflow and am curious about the team's perspective on this idea. Needless to say, I understand my suggestion might be oversimplifying these complexities, and—in open-source projects in particular—resources might be rather scarce and thus must be managed very carefully. Even if this means sacrificing otherwise desirable features.
@kerberizer It's a fair question, and I think you may have already guessed the answer. In point releases we have:
- Linux kernel compatibility fixes
- Code cleanups / documentation updates / test case updates
- New features
- Bug-fixes
- FreeBSD updates
- Compiler fixes/workarounds
We can of course pick one category to optimize for and get quicker releases for that category. However we do that at the expense of slower releases for the other categories. Every ZFS user is going to have different ideas about what categories are the most important to them. A RHEL 8 user may care a lot about bug-fixes, but not about 6.x kernel compatibility. As a result, we have to split the difference and try to (imperfectly) target a release for all categories.
@tonyhutter
- New features
Doesn't this go against the very idea that patch release should be super duper stable? Adding any new code is potentially a instability exposure. Adding a new feature can be just multiplying that risk by a much bigger factor.
@devsk I'd say "new feature" is probably an overstatement. Historically, we're talking about minor improvements in non-critical areas. For example, an additional option to zdb to facilitate debugging, or a new module parameter which is useful but doesn't change the default behavior. This wouldn't include any significant/major features.
Thank you for taking the time to explain your processes and why you do what you do. And I can totally understand it.
btw a bit offtopic but interesting so I don't have to open an issue next time: Since #15842 has now been merged into main I would just wait until the official release comes out, but could it possibly be that OpenZFS has discontinued the testing repository for Fedora?
According to http://download.zfsonlinux.org/fedora-testing/ Fedora 34 is the last version for which staging packages were built. So there is no official way to get staging packages without building them yourself, right?
@Alexandero89 that's correct, you will need to build it from source or wait for 2.2.3 to get #15842. None of the testing repos have -staging code.
Testing repo in a nutshell:
- Fedora testing - occasionally been used over the years for -rc testing, but it's kind of an unused repo.
- EL testing - The 2.2.x branch.
The normal EL repo contains the 2.1.x branch.
I wonder why tagging 2.2.3 is taking so long when 2.2.1 was tagged a month after 2.2.0 on Nov 13th and 2.2.2 was tagged 2 weeks after 2.2.2 on Nov 28. Its been more than 2 months since 2.2.2.
Is there a showstopper blocking issue for 2.2.3? I thought 2.2.x is now a patchset series which would see very minimal changes and not break much, while 2.3 will take the new features etc. Is this not correct interpretation of release versions?
Fedora 38 is now shipping a 6.7.x kernel and this bug makes me nervous about running the update...
So #15842 is merged, but the tests did not pass competly. Seems that it's not a zfs problem but a github actions problem. Either there is a network connectivity issue or just the machine just ran out of disk space ( see github action)
Fedora 38 is now shipping a 6.7.x kernel and this bug makes me nervous about running the update...
maybe applying
echo 'zfs' > /etc/dnf/protected.d/zfs.conf
as guided in openzfs docs help?
Fedora 38 is now shipping a 6.7.x kernel and this bug makes me nervous about running the update...
maybe applying
echo 'zfs' > /etc/dnf/protected.d/zfs.conf
as guided in openzfs docs help?
Thanks, did not know this is possible.
The only consequence on my system, if there is no compatible zfs yet, is that a new kernel is installed that does not have a zfs. in other words, if you reboot the system and automatically use the new kernel, the system would no longer boot. however, if you have the grub selection with kernels, for example, you can simply select the second last kernel with zfs and everything will continue to work perfectly. of course this makes the kernel update pointless.
i simply use dnf exclude as long as there is no suitable zfs release:
sudo dnf upgrade --exclude="kernel*"
echo 'zfs' > /etc/dnf/protected.d/zfs.conf doesn't prevent installing a kernel (like 6.7.x) for which zfs-dkms can not build a module so on reboot all ZFS filesystems fail to mount. It just stops yum/dnf from uninstalling zfs itself.
So even with that, dnf upgrade (without exclude) is a bad idea right now :(
I haven't updated to Fedora 39, but expect that would be a even worse idea than updating the Fedora 38 kernel (from 6.6.x) right now :(
So even with that,
dnf upgrade(withoutexclude) is a bad idea right now :(
so at least for new installations (specifically netinstall) giving an option to select kernel version would be nice to implement?
So #15842 is merged, but the tests did not pass competly. .... ust the machine just ran out of disk space ( see github action)
Certainly seems like it just needs some TLC. Can someone poke it back to life ?
Lack of support for 6.7.x is starting to hurt people ...
So #15842 is merged, but the tests did not pass competly. .... ust the machine just ran out of disk space ( see github action)
Certainly seems like it just needs some TLC. Can someone poke it back to life ?
Lack of support for 6.7.x is starting to hurt people ...
Can confirm that a naive dnf upgrade led to an unusable ZFS pool. Same situation as it was when F39 was launched. Makes me consider to migrate to some in-kernel FS...
So #15842 is merged, but the tests did not pass competly. .... ust the machine just ran out of disk space ( see github action)
Certainly seems like it just needs some TLC. Can someone poke it back to life ? Lack of support for 6.7.x is starting to hurt people ...
Can confirm that a naive dnf upgrade led to an unusable ZFS pool. Same situation as it was when F39 was launched. Makes me consider to migrate to some in-kernel FS...
Same issue here. Thankfully still had a 6.6 Kernel installed that I could boot into.
Ha, found this thread the same way. Did a dnf update on Fedora 39 and after reboot my ZFS filesystems were gone. No biggy though, just rebooted into an older Kernel and pinned it for now.
And to be fair, had I bothered to scroll up a couple of screens I'd have seen the dkms autoinstall on 6.7.4-200.fc39.x86_64/x86_64 failed for zfs(10) message. Unlucky that dnf doesn't seem to exit with an error when a dkms build fails.
On the other hand, why doesn't zfs-dkms 2.2.2 conflict with kernel versions >= 6.7? The spec file lists some Requires but doesn't prevent my system from upgrading to an incompatible kernel version. Wouldn't a Conflicts: kernel > @[email protected] solve the issue? I feel like this question must have been asked before and I'm missing something very obvious.
