jimsalterjrs
Feature request: Ability to change snapshot name format
I would love the ability to change the snapshot name format.
It's a minor nitpick, but if you put these two snapshot names in to your terminal:
tank/officeshare@zfs-auto-snap_04-2020-03-01-1600
tank/officeshare@syncoid_usrbgofnas01_2020-05-01:20:42:59
...and then try to double-click each one to select it, the snapshot created by zfs-auto-snapshot automatically selects the entire name. The snapshot by sanoid only selects tank/officeshare@syncoid_usrbgofnas01_2020-05-01 because it runs into a colon.
I would love the ability to change the snapshot name format so date and time are separated by an underscore and the the individual fields are separated by dashes.
Hi,
This feature would be essential for my use case: I need to share snapshots via SMB (Samba) to Windows computers, which is currently impossible because the colon is a forbidden character. Currently, snapshot name is mangled by Samba, which makes it meaningless.
Regards, Yvan
@Yvan-Masson you can use the samba catia module to remap the illegal chars, see: https://github.com/jimsalterjrs/sanoid/issues/345#issuecomment-459660254
@phreaker0 Thanks for the tip! I have also just seen samba shadow_copy2 module which might not be trivial to setup but could do the job (https://github.com/jimsalterjrs/sanoid/issues/5#issuecomment-730660664).
It would be helpful to alter snapshot name to remove the seconds from the snapshot name. I have TrueNAS pulling Sanoid snapshots and TrueNAS does not allow seconds as a variable in schema name. So I have to hard code that as ":00", but often Sanoid second is at 01 which is annoying because we don't need seconds in a snapshot name.
I'm not sure what you mean by "TrueNAS pulling Sanoid snapshots." Can you elaborate?
Jim, be glad to explain it...
TrueNAS (previously known as FreeNAS) has the ability to PUSH out to other servers and/or PULL in from other servers; ZFS snapshots using its built in "Replication Tasks". Behind the scenes the SSH session happens and the normal ZFS send / receive happens.
When defining a Replication Task on TrueNAS to PULL snapshots created by sainoid on an Ubuntu 20.04.1 system, besides defining the source ZFS pool and dataset(s) to pull from I have to define a Naming Schema to parse the snapshot names to define a filter of which snapshots to import. They look like:
autosnap_%Y-%m-%d_%H:%M:00_hourly
autosnap_%Y-%m-%d_%H:%M:00_daily
autosnap_%Y-%m-%d_%H:%M:00_monthly
As you will notice, I have to hard code the ":00" for seconds as TrueNAS does provide a variable placeholder to represent the seconds. So I want to be able to configure Sanoid to not have seconds in the snapshot name. They are useless, Nobody is doing multiple snapshots per minute.
The problems happens when Saniod is 1 second late and the snapshot ends with ":01" TrueNAS can't see this snapshot and thus can not pull it. I only want accuracy to the minute, not the second.
Hope that was helpful.
According to TrueNAS docs at https://www.ixsystems.com/documentation/freenas/11.3-U3.2/tasks.html#replication-creation-wizard, TrueNAS naming schema uses FreeBSD strftime(), which links to manpage here: https://www.freebsd.org/cgi/man.cgi?query=strftime
strftime() supports %S for seconds in the same format Sanoid uses. From where I'm sitting, if TrueNAS doesn't support %S, that's a TrueNAS bug which should be reported there. (They really, REALLY ought to support wildcard use as well, but that's a slightly different kettle of fish I suppose.)
You're right, looking back I did a typo of "%s" not "%S". So I can make it work, tested this morning it works.
I would not have opened a TrueNAS bug to fight for the ability to have seconds when I'm trying to get rid of them. To me it makes no sense to have seconds in the schema name when Sanoid is doing snapshots at most every 15 minutes.
My request still stands to be able to alter the Sanoid snapshot name format to get rid of the seconds.
Last comment, I noticed this in the defaults conf:
# for snapshots shorter than one hour, the period duration must be defined
# in minutes. Because they are executed within a full hour, the selected
# value should divide 60 minutes without remainder so taken snapshots
# are apart in equal intervals. Values larger than 59 aren't practical
# as only one snapshot will be taken on each full hour in this case.
I take that to mean there isn't a way to get more than 1 snapshot per minute, so accuracy down to the seconds is irrelevant. Not seeing the value of it being in the name.
@reefland In some cases the seconds field is annoying (i.e. when trying to integrate snapshots with Samba's VFS module), but I don't think chopping it off would be a good idea. Being precise is good. But I totally agree the snapshot names should be modifiable. I can't tell you how much I despise colons in the snapshot name. When double-clicking on them in a Linux terminal to highlight text, it stops at colons meaning I have to click and drag to select instead of double-clicking. It's a small thing, but it's annoying.
zfs-auto-snapshot does names like this: tank@zfs-auto-snap_hourly-2021-02-02-0217. Easily double-clickable to select. ;)
zfs auto snapshot as well as TrueNas does not include seconds in the snapshot name. :)
One last request to be able to change snapshot name. All my _monthly snapshots failed to replicate to TrueNAS this month because the snapshot name was 1 seconds ahead of the snapshot creation time.
If I could remove the seconds from the snapshot name I could resolve this issue. Perhaps sanoid naming snapshots that don't match the creation time is real bug, not sure.
Here is the failure from TrueNAS trying to pull in this snapshot:
* Replication "tank/docker_projects -> main/docker/docker_backups" failed: zettarepl: zfs send PID is 852589
warning: cannot send 'tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly': not an earlier snapshot from the same fs
Replication cannot continue because existing snapshot
autosnap_2021-03-01_00:00:01_hourly is newer than
'tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly', but has an older date
in the snapshot name. To resolve the error, rename
'tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly' with a date that is older than
autosnap_2021-03-01_00:00:01_hourly or delete snapshot
'tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly' from both the source and destination..
Snapshot name is one second newer than snapshot creation time:
$ zfs get creation tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly
NAME PROPERTY VALUE SOURCE
tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly creation Mon Mar 1 0:00 2021
Rename snapshot to match:
sudo zfs rename tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly tank/docker_projects/certbot@autosnap_2021-03-01_00:00:00_monthly
# Verified name matches creation time.
$ zfs get creation tank/docker_projects/certbot@autosnap_2021-03-01_00:00:00_monthly
NAME PROPERTY VALUE SOURCE
tank/docker_projects/certbot@autosnap_2021-03-01_00:00:00_monthly creation Mon Mar 1 0:00 2021 -
The replication error clears...
The following alert has been cleared:
* Replication "tank/docker_projects -> main/docker/docker_backups" failed: zettarepl: zfs send PID is 852589
warning: cannot send 'tank/docker_projects/certbot@autosnap_2021-03-01_00:00:01_monthly': not an earlier snapshot from the same fs
Replication cannot continue because existing snapshot....
Now I have to find all snapshots matching "@autosnap_2021-03-01_00:00:01_monthly" and rename them to be 1 second sooner.
I strongly suggest you file a bug with TrueNAS re: its odd refusal to replicate snapshots based on their names. That's not a ZFS thing... that's a TrueNAS thing.
I've left a message in the form, we'll see if it gets any traction. I agree it's not a ZFS thing.
But if I could just remove that seconds field from the snapshot name, life would be simpler :)
BTW - you look good in that coder robe. :)
I would love to be able to customize the snapshot names--maybe a prefix and/or suffix that gets run through through strftime. i.e.:
SANOID_SNAPSHOT_PREFIX='my-sanoid-snapshots_'
SANOID_SNAPSHOT_SUFFIX='%Y-%m-%d_%H-%M-%S'
I'd love to have the ability to customize snapshot names too. When working with multiple ZFS filesystems and number of snapshots having nice naming conventions that suits your tasks helps a lot. Personally I'd prefer the ability to get rid of 'autosnap_' part and have the periodic name first - like zpool/zfs@monthly-2021-03-01 It would help a lot when one wants to get all dailies for example - you just "zfs list -t snap |grep zpool/zfs@daily" Of course, still possible with a bit of regexping. But have it simplified would be nice.
yeah I would love the ability to set names as well. Sure we can can hack the code but it's a lot of effort… I'd like to be able to do prefix-daily/week/etc-time, or similar. It makes it even better with samba shadow copies, because you can use wildcards before the date but not after
+1 for customizable snapshot name. It's useful for multiple reasons mentioned above.
Here is an example of renaming a snapshot to make it easier to handle with Samba's vfs_shadow_copy2.
https://github.com/kimata/sanoid/commit/5d92ede545138fa1af63e87e7ccdec313f46d297
After applying the above changes, the following settings in smb.conf should work.
shadow:format = _UTC_%Y-%m-%d_%H:%M:%S
shadow:snapprefix = ^autosnap_\(yearly\)\{0,1\}\(monthly\)\{0,1\}\(weekly\)\{0,1\}\(daily\)\{0,1\}\(hourly\)\{0,1\}\(frequently\)\{0,1\}
shadow:delimiter = _UTC_
shadow:localtime = no
For anyone needing this functionality, if you're willing to test zfs-autobackup, its available in v3.1.1.
For anyone needing this functionality, if you're willing to test zfs-autobackup, its available in v3.1.1.
I just checked and couldn't find any hint on the page you linked about snapshot naming format.
Sorry, I have yet to update the README file, i'll do that later this week.
If you have v3.1.1 you can find it in the --help
--snapshot-format FORMAT
Snapshot naming format. Default: {}-%Y%m%d%H%M%S
--property-format FORMAT
Select property naming format. Default: autobackup:{}
--hold-format FORMAT Hold naming format. Default: zfs_autobackup:{}
{} will be replaced by the backup name.
The date/time formatting/parsing is done via pythons time.strftime() and time.strptime()
--snapshot-format FORMAT
Is this smart enough to handle a change in format?
Partially: It wont recognize the old snapshots as its own anymore. (so it will never delete those, you have to manually do that)
But it can continue from any common snapshot, so it can just continue on.
Would be great if sanoid supported this. Would make it easy to purge snapshots that were not created by sanoid itself. I'll try to hack sanoid into doing this. ;-) These lines look interesting:
- my $snapname = "autosnap_$datestamp{'sortable'}_$type";
- if ($snapname =~ /^autosnap/) {
- $datestamp{'sortable'} = "$datestamp{'noseconds'}:$datestamp{'sec'}";
Hi @jimsalterjrs as already stated by different people, it would be great if Sanoid was able to give snapshots a flexible custom name. But even when you are not seeking for the complete solution, it would be great to change just one thing in the naming scheme or to give an option to just switch between them.
Depending on the OS you use, some symbols/characters are reserved for the specific system and cannot be used within file/folder names. For Windows one of these charecters is the colon " : "
Unfortunately Sanoid uses exactly this character to separate hours : minutes : seconds in the filename.
Due to this, Windows is not able to display these snapshots in the .zfs/Snapshots folder when viewing with the Windows File Explorer and they cannot be seen in the "Previous Version" function of Windows.
So unfortunately Sanoid is useless for Windows users - and this is a big user base you should not ignore.
Looking at your code I could find the related function at about line 1017 within the "sub get_date" block.
One could simply exchange the colons there against dots and voila, Windows can immediately see the files in the .zfs/Snapshots folder.
To go one step further and enable the snapshots to be available at the Windows Previous Files function, they must be presented to to Windows follow this naming scheme: @GMT-2022.03.22-11.27.11.
To achieve this the name of the snapshot have to/can be converted by Samba to the a.m. Windows-readable format by using a configuration in the smb.conf : vfs_shadow_copy2 and the option shadow:format.
So we see that it is NOT necessary to exactly name the snapshot this way.
But again, though on top of this there even IS a way (look for catia) to convert reserved characters, this DOES NOT work when you have to convert from : to . (and vice versa)!
So this fault in naming the snapshots has to be corrected at the time the snapshot is being created.
For an explanation see here: https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html
shadow:format = format specification for snapshot names This is an optional parameter that specifies the format specification for the naming of snapshots in the file system. The format must be compatible with the conversion specifications recognized by str[fp]time. Default: shadow:format = "@GMT-%Y.%m.%d-%H.%M.%S"
Of course, I could change these things manually in the code. But that leads to another thing I have to remember when I setup a new system or when there is an update to my base system which overrides this change - I think you know, this can be a PITA. So for all people the best solution would be to have this change implemented at a central point and this is here.
Thanks and have a nice day! ;-)
