zfs icon indicating copy to clipboard operation
zfs copied to clipboard

zdb: add -B option to generate backup stream

Open robn opened this issue 2 years ago • 9 comments

Motivation and Context

We were working on a data recovery for a client, and the dataset name table was damaged to the extent that referencing a snapshot we needed, or asking for its name, would panic ZFS.

On further investigation, we found that dmu_send_obj didn't care at all about the name, so we just needed a way to arrange for it to be called.

Description

This adds a -B option, which will produce a standard backup stream from the given pool/objset-id. An optional arg can set the flags.

# zdb -B lucy/34760 Lecw | ./zstream dump
BEGIN record
	hdrtype = 1
	features = 3420000
	magic = 2f5bacbac
	creation_time = 63dc5e6c
...

Sponsored-By: Klara, Inc.

How Has This Been Tested?

Only local testing on the specific feature. Test suite should get anything of interest.

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Performance enhancement (non-breaking change which improves efficiency)
  • [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • [x] Documentation (a change to man pages or other documentation)

Checklist:

  • [x] My code follows the OpenZFS code style requirements.
  • [x] I have updated the documentation accordingly.
  • [x] I have read the contributing document.
  • [ ] I have added tests to cover my changes.
  • [ ] I have run the ZFS Test Suite with this change applied.
  • [x] All commit messages are properly formatted and contain Signed-off-by.

robn avatar Mar 17 '23 00:03 robn

@robn - In general - this is a really cool idea - and also the implementation looks straightforward.

Could you rebase the thing? It's needed for the new testing workflow... So we see, it doesn't break sth. I will approve this next week, when the testings are run.

mcmilk avatar Mar 17 '23 21:03 mcmilk

Could you rebase the thing? It's needed for the new testing workflow... So we see, it doesn't break sth. I will approve this next week, when the testings are run.

Oops! That's a thing I normally just do before a new push, not sure why I missed it this time. Done! Thanks for pointing it out!

robn avatar Mar 17 '23 22:03 robn

	if (write(1, buf, len) < 0)
		return (errno);

Short writes may occur.

Low-power avatar Mar 20 '23 16:03 Low-power

Short writes may occur.

Yeah, fair. I've pushed something that should be a little more resilient.

robn avatar Mar 20 '23 19:03 robn

Don't use arithmetic operation on a void * directly, cast it to char * first, like:

Yeah, silly me.

Consider using some C99 features to reduce scpoe of some variables, and make initialization looks better:

Agreed. I spend a lot of time in codebases where old conventions are still the rules, and a lot of ZFS still looks like that, so I don't think much about it. But reviewer wins here :)

Thanks!

robn avatar Mar 21 '23 02:03 robn

Lasty in this comment:

/* Write the data out, handling short reads and signals. */

I think the code means short writes not reads.

Low-power avatar Mar 21 '23 04:03 Low-power

When trying out this feature, I accidently runs it without redirecting, thus trashed my terminal...

May be it is better to have a isatty(STDOUT_FILENO) check just like in zfs-send(8).

Low-power avatar Mar 21 '23 05:03 Low-power

I think the code means short writes not reads.

Sigh, I'm really having a day of it today. Thanks, fixed.

May be it is better to have a isatty(STDOUT_FILENO) check just like in zfs-send(8).

Great suggestion. Done!

robn avatar Mar 21 '23 06:03 robn

@behlendorf thanks for the heads up, I missed those - I thought they were obviously false positives, with all the churn happening in the tests at the time.

I should have time to get back to this in the next week or two. Cheers.

robn avatar Apr 10 '23 23:04 robn

Not quite done with this yet, still have to add a test for it and I want to update the docs a little bit to say that it can handle sending whole real datasets, not just snapshots (which it obviously can, but I hadn't come at it from that angle, and then we used it for that and won the day :astonished:).

robn avatar May 30 '23 21:05 robn

@robn sounds good.

behlendorf avatar May 30 '23 22:05 behlendorf

@behlendorf good to go (pending remaining tests). I added a commit to expose a function I need; up to you if you would prefer to squash it or not. Cheers!

robn avatar Jun 04 '23 04:06 robn

Merged, I opted to keep the commits split.

behlendorf avatar Jun 05 '23 18:06 behlendorf

Thanks all!

robn avatar Jun 05 '23 19:06 robn