bees
bees copied to clipboard
Zygo
kernel 5.2 btrfs send vs dedupe mitigation requires new bees behavior
The fix in https://github.com/torvalds/linux/commit/62d54f3a7fa27ef6a74d6cdf643ce04beba3afa7 prevents the send from failing (or worse) if dedupe is running at the same time.
bees will get an EAGAIN error from dedupe if there is a send in progress, but bees doesn't yet do any of the "rescheduling" proposed in the mailing list discussion behind the patch. This generates a lot of noise in the log, and bees currently just skips the extents. If the snapshot is deleted after the btrfs send, then we didn't need to dedupe it anyway; however, if the snapshot is not deleted after the btrfs send, the duplicate extent references that are skipped during send will not ever be deduped.
Ideally bees would pause the subvol crawler when we see EAGAIN in dedupe, and unpause when we restart crawlers (at 10-transid intervals). If the send is still in progress, the crawler will be immediately paused again. Presumably at some point the send will be done, and the crawler can finish deduping the subvol.
Originally posted by @Zygo in https://github.com/Zygo/bees/issues/79#issuecomment-500090196
I think kernel 5.3 is going to be LTS and will be picked up by distributions soon. Should working on this be prioritized?
It's currently a quality-of-implementation issue. If you were using --workaround-btrfs-send before 5.3, it still works on 5.3.
The "deferred subvol scan" feature is an alternative to --workaround-btrfs-send, so it's a new feature with a minimum kernel dependency on 5.3.
Under what scenarios --workaround-btrfs-send is still needed for Kernel 5.11.0 or later
Under what scenarios --workaround-btrfs-send is still needed for Kernel 5.11.0 or later
None.
Well, OK, one: you can't start a btrfs send if there is a dedupe call running on the subvol at the time. --workaround-btrfs-send will prevent bees from interfering with send startup.
Makes sense. To clarify:
If I'm running Kernel 5.11.0 or later (Debian/Ubuntu), then as long as I have --workaround-btrfs-send, I am OK using readonly snapshots.
send-receive should work as expected (it might happen that during the send, any FS writes/updates wont be processed by bees, but after the send, at a later bees run, those FS writes/updates pending bees processing, would then be processed by bees incremental)