openzfs
zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
If the fields to be listed and sorted by are constrained to those populated by dsl_dataset_fast_stat(), then zfs list is much faster, as it does not need to open each objset and reads its properties.
A previous optimization by Pawel Dawidek (0cee24064a79f9c01fc4521543c37acea538405f) took advantage of this to make listing snapshot names sorted only by name much faster.
However, it was limited to -o name -s name, this work extends this optimization to work with:
- name
- guid
- createtxg
- numclones
- inconsistent
- redacted
- origin and could be further extended to any other properties supported by dsl_dataset_fast_stat() or similar, that do not require extra locking or reading from disk.
This was committed before (9a9e2e343dfa2af28bf7910de77ae73aa006de62), but was reverted due to a regression when used with an older kernel.
If the kernel does not populate zc->zc_objset_stats, we now fallback to getting the properties via the slower interface, to avoid problems with newer userland and older kernels.
Signed-off-by: Allan Jude [email protected] Sponsored-by: Klara Inc.
Motivation and Context
Make zfs list -H -o name,guid faster
This was committed back in December of 2021, but reverted in January due to causing the userspace tools to crash if run against an older kernel. This updated version resolves this issue.
Description
Expand the list of properties that trigger 'simple' mode for the zfs list IOCTL
The addition of createtxg to the list of optimized properties also means the default sort order no longer requires fetching properties. (this original advantage has been somewhat overcome by 3a1ce4914172ce4c1e39123cd31b1e5245765a5e which was committed in July)
How Has This Been Tested?
Comparing the speed of zfs list of 20000 datasets (304% improvement):
time zfs list -H -o name -s name > /dev/null
or
time zfs list -H -o name,guid > /dev/null
x patched_list_name.time
+ unpatched_list_name.time
N Min Max Median Avg Stddev
x 20 1.35 1.37 1.36 1.361 0.0055250625
+ 20 5.5 5.52 5.5 5.5055 0.0068633274
Difference at 95.0% confidence
4.1445 +/- 0.00398763
304.519% +/- 0.777466%
(Student's t, pooled s = 0.00623023)
as well as zfs list -t snapshot for 10 datasets each with 2000 snapshots (20000 snapshots), the improvement was 378%
time zfs list -H -t snapshot -o name,guid > /dev/null
x patched_list_snapguid.time
+ unpatched_list_snapguid.time
+------------------------------------------------------------+
N Min Max Median Avg Stddev
x 20 1.56 1.58 1.57 1.57 0.0056195149
+ 20 7.49 7.54 7.515 7.517 0.015252265
Difference at 95.0% confidence
5.947 +/- 0.00735649
378.79% +/- 0.891556%
(Student's t, pooled s = 0.0114937)
To confirm that these changes did not disturb the original enhancement:
time zfs list -H -t snapshot -o name -s name > /dev/null
x patched_list_snapname.time
+ unpatched_list_snapname.time
+------------------------------------------------------------+
N Min Max Median Avg Stddev
x 20 1.49 1.52 1.5 1.5015 0.0074515982
+ 20 1.49 1.51 1.5 1.4975 0.007163504
No difference proven at 95.0% confidence
Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Performance enhancement (non-breaking change which improves efficiency)
- [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation (a change to man pages or other documentation)
Checklist:
- [x] My code follows the OpenZFS code style requirements.
- [ ] I have updated the documentation accordingly.
- [x] I have read the contributing document.
- [ ] I have added tests to cover my changes.
- [ ] I have run the ZFS Test Suite with this change applied.
- [x] All commit messages are properly formatted and contain
Signed-off-by.
This reminds me of the following patch that I never upstreamed:
https://github.com/ryao/zfs/commit/b76b403435ca032091d49fde66e9d2fb4b0b7a1b
I probably should take some time to rebase it. It is a nice performance win.
