Longname: files/directories name upto 1023 bytes

[tuxoko]

Motivation and Context

Description

This patch adds the ability for zfs to support file/dir name up to 1023 bytes. This number is chosen so we can support up to 255 4-byte characters. This new feature is represented by the new feature flag feature@longname.

A new dataset property "longname" is also introduced to toggle longname support for each dataset individually. This property can be disabled, even if it contains longname files. In such case, new file cannot be created with longname but existing longname files can still be looked up.

Note that, to my knowledge native Linux filesystems don't support name longer than 255 bytes. So there might be programs not able to work with longname.

Note that NFS server may needs to use exportfs_get_name to reconnect dentries, and the buffer being passed is limit to NAME_MAX+1 (256). So NFS may not work when longname is enabled.

How Has This Been Tested?

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Performance enhancement (non-breaking change which improves efficiency)
• [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• [ ] Documentation (a change to man pages or other documentation)

Checklist:

• [x] My code follows the OpenZFS code style requirements.
• [ ] I have updated the documentation accordingly.
• [x] I have read the contributing document.
• [x] I have added tests to cover my changes.
• [x] I have run the ZFS Test Suite with this change applied.
• [x] All commit messages are properly formatted and contain Signed-off-by.

[tonyhutter]

@tuxoko sorry this PR has sat on the sidelines for so long. Can you rebase on master and I can take a look?

[tuxoko]

@tonyhutter rebased

[jumbi77]

@tuxoko Can I may ask for another rebase and @tonyhutter for review? Get this done would be great! In any case much thanks.

[tonyhutter]

Note that, to my knowledge native Linux filesystems don't support name longer than 255 bytes. So there might be programs not able to work with longname.

That seems to be the case. I'm unable to write a >256B filename on Fedora 40 using this PR :

/tank$ touch $(printf 'a%.0s' {1..255})
/tank$ touch $(printf 'a%.0s' {1..256})
touch: cannot touch 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': File name too long

So what would be the use case for 1024B filenames if it's not supported by Linux? Does it work in FreeBSD?

[user-566]

So what would be the use case for 1024B filenames if it's not supported by Linux? Does it work in FreeBSD?

I might be misunderstanding your question, but Windows supports >255B filenames, which I assume represents a large portion of clients connected to TrueNas systems. Lots of discussion on this in https://github.com/openzfs/zfs/issues/13043

[Haravikk]

So what would be the use case for 1024B filenames if it's not supported by Linux? Does it work in FreeBSD?

IIRC the limit might be 255 characters, including multi-byte characters, but the current/historic ZFS limit is 255 bytes, so multibyte characters will eat into that. For example if you're using two-byte characters your limit is only 127 characters (254 bytes). This is also the case on macOS, which is what caused me to notice it as I was handling a lot of files with long, multibyte names at the time that worked fine on HFS+/APFS but not on ZFS.

There are a bunch of filesystems now that support either 255 characters, or have even less strict limits (so they'll support whatever the OS' limit(s) are).

It's also one of those classic chicken and egg problems – if ZFS keeps a limitation because it matched one on Linux, then Linux is less likely to increase that limit because a major filesystem doesn't support it etc.

IMO it's better for a filesystem to support any size that it doesn't have a good technical reason not to (due to overhead or whatever), that way it's not the part that's holding something back. Though it makes sense to have it tunable for those that need to guarantee portability between more and less limited OSes.

[tuxoko]

Note that, to my knowledge native Linux filesystems don't support name longer than 255 bytes. So there might be programs not able to work with longname.

That seems to be the case. I'm unable to write a >256B filename on Fedora 40 using this PR :

/tank$ touch $(printf 'a%.0s' {1..255})
/tank$ touch $(printf 'a%.0s' {1..256})
touch: cannot touch 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': File name too long

So what would be the use case for 1024B filenames if it's not supported by Linux? Does it work in FreeBSD?

@tonyhutter Did you enable both the pool feature and turn on the property on the dataset?

The kernel VFS layer itself doesn't impose a 255 limit on filename as long as the "path" length fits in 4k page. As you can see the unit tests are passed in the built bot, there shouldn't be any problem creating longname file with common shell command. Any program that rely only on syscall to check file name length should work fine. We have no problem running this with samba. The only notable issue if nfs server may need to call exportfs_get_name, and it has a hard coded buffer size of 256.

$ touch $(printf 'a%.0s' {1..255})
$ touch $(printf 'a%.0s' {1..256})
$ ls
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

[tonyhutter]

@tuxoko thanks, you were right, I wasn't enabling the longname dataset feature. Its working fine for me natively in Linux. :+1:

[tonyhutter]

I'm pretty happy with this - if you want to rebase it I can approve it.

[robn]

I had a few of drive-by questions/thoughts. Note I don't consider these blockers; just thinking about the problem space.

Is it worth storing the max filename length somewhere on the dir ZAP (extended attribute? somewhere on the bonus?) so that we can increase it further in the future without needing another feature flag etc?

I assume we're happy leaving leaving interpretation of the bytes in the filename as out of scope (ie, normal utf8only, normalization, casesensitivity, etc)?

Is there any value in a kind of "compatibility" option, so that older implementations can still use these dirs? I personally think not; it would be a lot of complexity for perhaps very little advantage. And its tricky to do well; at least, I remember FAT32 long filenames being surprisingly difficult to get right :)

[behlendorf]

@tuxoko the updated ABI files look like they contain a bunch on unrelated changes (and they they pass). If you remove the ABI file changes, then force update the PR you should be able to download new ABI files from the "checkstyle" builder summary section with the minimal changes required. e.g. generated with the exact same version of libabigail.

[behlendorf]

Updated ABI files available: https://github.com/openzfs/zfs/actions/runs/11022167809?pr=15921

[tuxoko]

Updated with the abi files and freebsd compile fix. Hopefull we can get a freebsd run now.

A couple of notable new changes, I added check in fzap_checkname to make sure we don't accidentally use longname for non directory zap. Also remove readonly compat for SPA_FEATURE_LONGNAME, as it might cause problem.

[tuxoko]

Update, fixed compile error and a typo in test. Also, it turns out freebsd has 255 limit in vfs layer. https://github.com/freebsd/freebsd-src/blob/01eb635d12953e24ee5fae69692c28e4aab4f0f6/sys/kern/vfs_lookup.c#L1132 So this won't actually work on freebsd, so I moved the unit tests back to linux only.

[behlendorf]

@tuxoko looks good. The CI failures here all look to be unrelated. I can handle resolving the trivial conflict in the merge.

[tschoening81]

Maybe I missed it, but could someone please explain how the implementation actually works currently? The PR is just to large and in-depth for me to answer the questions I have. Thanks!

8.3-approach

In some other related issues a suggestion for example was to follow the Windows 8.3 naming scheme for backwards compatibility reason, storing the real long name additionally in some xattr. But that would have influence on tools looking at filenames, which I don't see documented in the feature description currently.

So, there's no special backwards compatible workaround and a long filename is simply stored in a larger buffer, but still as the one and only filename? Tools always see one and the same name only and need to support a possibly longer name?

Performance impact

OTOH, if longer names are stored sperately with/without an approach like 8.3 for the original name, that might influence performance. Such an approach might be worth it mentioning a possible performance impact in the docs as well in my opinion. As I don't see it in the docs, that is not the implemented approach or do you just disagree the need of documenting those things?

What I have in mind is e.g. encryption, which RSYNC based filesystem iteration a lot slower in my setup as well. There are some discussions about other people having similar observations and that it might just be like that because of how encryption works. But that slowdown was somewhat surprising to me and I would have liked a mention of it in the docs beforehand.

Read-only

There's the following comment:

Also remove readonly compat for SPA_FEATURE_LONGNAME, as it might cause problem.

https://github.com/openzfs/zfs/pull/15921#issuecomment-2380343195

How is that related to the current docs?

This property can be disabled even if it contains longname files. In such case, new file cannot be created with longname but existing longname files can still be looked up.

https://openzfs.github.io/openzfs-docs/man/master/7/zpool-features.7.html#longname