btrfs-progs icon indicating copy to clipboard operation
btrfs-progs copied to clipboard

Feature request: new mount option and property 'datasum=force'

Open Forza-tng opened this issue 2 years ago • 1 comments

There are lots of applications that choose nodatacow as default for files to improve performance for their users.

However, since nodatacow implies nodatasum, it means that we lose integrity on these application files. Also in case Btrfs RAID1/10 is used, btrfs cannot determine which copy is good if there is data corruption.

I would like to have a new mount option datacow=force or datasum=force that can be used on subvolume context to prevent applications from knowing better than an informed users' choices.

A complement would be a btrfs property that can be set on files and directories that has the same effect.

https://btrfs.readthedocs.io/en/latest/Administration.html

Forza-tng avatar Jun 18 '22 11:06 Forza-tng

I second this request. From my perspective, a filesystem-wide mount option would suffice.

I am also very much annoyed by software that tries to be "smart" and disables corruption resilience, which has been the most important reason for choosing dup/raid1/... profiles in the first place for me.

manxorist avatar Jun 18 '22 11:06 manxorist

There are lots of applications that choose nodatacow as default for files to improve performance for their users.

Do you have any examples? I didn't know this is common.

amstan avatar Oct 29 '22 17:10 amstan

Debian postgresql packages used to install with +C set on /var/lib/postgresql/*/* during installation. I had to set up post-install automation to turn it off.

Zygo avatar Oct 29 '22 19:10 Zygo

There are lots of applications that choose nodatacow as default for files to improve performance for their users.

Do you have any examples? I didn't know this is common.

For example systemd/journald and libvirtd.

Forza-tng avatar Feb 25 '23 21:02 Forza-tng

I would honestly prefer this be a fs wide option (as in, a mount option or a sysfs tunable that applies to the entire fs), not per subvolume or per-directory attribute or property. It's already a management nightmare otherwise. I made a request for this here on the btrfs mailing list describing the downsides in detail that I run into with btrfs as an admin.

Many container platforms (like LXC/LXD) make it increasingly complicated since each distro container means yet another set of configs you need to be wary of. Since datacow is already a mount option, and is default, I think having something like datacow=always or datacow=forced would be appropriate, much like we have discard and discard=async currently.

I recognize something like this would likely break swap files, that's a cost I'm willing to accept, especially since I especially want this for using with Btrfs RAID where said swap files can't be used anyway.

jsebean avatar Mar 02 '23 00:03 jsebean

From https://libvirt.org/formatstorage.html: Some pools support optional features:

<features>
  <cow state='no'>
</features>

Valid features are: cow Controls whether the filesystem performs copy-on-write (COW) for images in the pool. This may only be set for directory / filesystem pools on the btrfs filesystem. If not set then libvirt will attempt to disable COW on any btrfs filesystems. Since 6.6.0.

😶

~~I need to have a conversation with the libvirt folks.~~ This was reverted in 6.7.0 but I guess the docs were never updated:

Fix logic in setting COW flag on btrfs When COW is not explicitly requested to be disabled or enabled, then libvirt should do nothing on non-BTRFS file systems.

richardm1 avatar Mar 08 '23 21:03 richardm1