Add/enhance tools to switch checksums

[kdave]

There's an experimental support for switching the checksum algorithm, implemented as btrfstune --csum NEWCSUM. It is now in usable state but thera are some some important sanity checks missing. The implementation still has drawbacks: the change happens at once, old checksums are not verified.

[megapro17]

Is there any news about it? Very interesting feature, definitely deserves a bit of attention

[kdave]

No news regarding the missing functionality, but it's been merged and released in 5.17, still behind the experimental build option.

[kdave]

Todo:

• [ ] add safety checks, ie. add a bit like UUID rewrite and make sure that it can't run at the same time
• [ ] implement hidden root of the new checksum tree and use it as temporary storage until the conversion is complete
  • add a temporary key item
  • do not drop the original csum tree until it's completed
• [ ] make the operation interruptible and restartable
• [ ] add way to drop the hidden new root

[kdave]

Updated in devel but still not complete.

Previous implementation copied the UUID conversion which was not a good
base for the checksum conversion so it left out basically all trees
except extent and checksum.

This update adds the base for the required safety features:

- let the old csum tree intact until the full conversion is done (ie.
  all data are still verifiable)
- add on-disk status tracking item, this should keep the from/to
  checksum conversion, last generation to catch potential updates of the
  underlying filesystem if conversion is interrupted and the filesystem
  mounted
- convert most of the fundamental trees, the subvolumes, tree log and
  relocation trees are not converted
- trees are converted in-place to avoid potentially running out of space
  but this might be better done by transaction protection with a
  temporary tree

Known issues:

- not all trees are converted
- not all checksums are correctly inserted into the new tree and reading
  the files leads to EIO

[StatusCode404]

Just wondering if there was a goal time-frame on when we could expect this kind of functionality?

It would be nice to have it sooner rather than later.

[adam900710]

For the current devel branch, you can enable the experimental feature and get a working btrfstune to use --csum to change checksum.

[kdave]

Just wondering if there was a goal time-frame on when we could expect this kind of functionality?

All the key parts have been implemented, we need to test the intermediate states and resume and verify the usability. Right now it's a single option but this is not sufficient so it may need to be promoted to the planned new btrfstune-equivalent command group. The 6.4 release is too close to finish that so 6.5 at the earliest.

[jshand2015]

I have an idea. before changing checksums, it would be a good idea to do a full disk scrub and fix any errors that are found. after full success, then you can change the checksum and rescrub

[adam900710]

Scrub is not enough, it doesn't do cross reference checks.

btrfs check --check-data-csum would be much better.

[jshand2015]

Scrub is not enough, it doesn't do cross reference checks.

btrfs check --check-data-csum would be much better.

oh, ok. that's good there is an option to do that. while this suggestion isn't relevant to this issue. would scrub be able to gain that function in future?

[adam900710]

oh, ok. that's good there is an option to do that. while this suggestion isn't relevant to this issue. would scrub be able to gain that function in future?

Unfortunately no. The full fsck functionality requires too many cross-ref checks, which is too big a burden for the kernel. Thus scrub is only limited to basic metadata checks.

[kdave]

The checksum change will be promoted out of experimental build after we have the separate command group for tune.