ZFS on MacOS 14+ should support new special file/dir flags (st_flags bits)

[captain-haddock17]

Related to new security policies and SIP protection, MacOS Mojave 10.14 and upwards are more strict on accessing some certain resources and files in ~/Library folders.

Issue

Seems that ZFS on OSX (v1.9.4) can not handle certain files/directories that are now protected by special flags.

Impact

This lead to abnormal behavior or errors of some apps trying to access files that are now secretly protected.

Here are some of these apps having issues handling this new security info on files stored on a ZFS filesystem:

• Mail.app
• rsync (not verified)

Some Folders concerned At present (2020-04) on Mojave 10.14.6/ Darwin 18.7.0 / root:xnu-4903.278.28~1

• ~/Library/Containers
• ~/Library/Mail
• ~/Library/Caches (?)

Result in operation not permitted when trying to read (like sudo find / -type f -exec grep -H something "{}" ";" )

• ~/Library/Caches/VoiceTrigger/SATUpdateNewerZone
• ~/Library/Containers/com.apple.mail/Data/DataVaults
• /private/var/db/fpsd
• /private/var/db/ConfigurationProfiles/Store
• /private/var/folders/*

and (maybe) Unable to quarantine error of Mail.app preventing to show-up the main Mail window (and emails ...), see issue #752

some clue

New filesystem flags, declared in /usr/include/sys/stat.h

_DATAVAULT    0x00000080      /* entitlement required for reading */
                              /* and writing */

Ref code :

/usr/include/sys/stat.h in (one of)

• /Applications/Xcode.app//Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/
• /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/
• /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/

Some articles on the web:

• The Eclectic Light Company No Entry : access controls in Mojave
• StackOverflow Explanation of st_flags bits
• Unix & Linux macOS Mojave Directory Permissions
• Ubuntu (french) macOS Autorisations de répertoire Mojave

[lundman]

This is interesting, I will have to study it in detail

[cyb3rz3us]

Correcting the Eclectic Light link error above:

No Entry ⛔️: access controls in Mojave (https://eclecticlight.co/2018/10/25/no-entry-%e2%9b%94%ef%b8%8f-access-controls-in-mojave/)

[captain-haddock17]

Thanks cyb3rz3us!

Some other tips to find out this special file flag datavault that ZFS seems not to handle for now with ls -lO

Scanning the my whole ~/Library, I found only 2 files (Mojave 10.14.6)

• ls -lOe@ ~/Library/Containers/com.apple.mail/Data/

drwx------@ 2 william staff datavault 64 30 déc 19:23 DataVaults com.apple.quarantine 19 com.apple.rootless 4

sudo xattr -l ~/Library/Containers/com.apple.mail/Data/DataVaults/

com.apple.quarantine: 0082;00000000;Mail; com.apple.rootless: Mail

• ls -lOe@ ~/Library/Caches/VoiceTrigger

drwxr-xr-x@ 2 william staff datavault 64 13 avr 11:10 SATUpdateNewerZone com.apple.rootless 10

sudo xattr -l ~/Library/Caches/VoiceTrigger/SATUpdateNewerZone

com.apple.rootless: CoreSpeech

[captain-haddock17]

Test case

• Copying DataVaults directory from APFS to ZFS

Conditions:

• SIP disabled (just for positive testing purpose)
• ZFS xattr=sa
• ZFS com.apple.mimic_hfs=on
• ~/Library = /User/myuser/Library is on startup APFS disk
• /Volumes/zDisk/Users/myuser is on startup ZFS disk (internal, virtual)

Take info

1. ls -leO@ ~/Library/Containers/com.apple.mail/Data

drwx------@ 2 myuser staff datavault 64 17 mar 09:12 DataVaults com.apple.quarantine 19 com.apple.rootless 4

Take référence on APFS filesystem

2. cp -av ~/Library/Containers/com.apple.mail/Data/DataVaults /Users/myuser/Documents

cp: /Users/admin/Library/Containers/com.apple.mail/Data/DataVaults: unable to copy extended attributes to /Users/myuser/Documents/DataVaults: Operation not permitted /Users/myuser/Library/Containers/com.apple.mail/Data/DataVaults -> /Users/myuser/Documents/DataVaults

Curiously has a ... unable to copy extended attributes... error/warning

3. `ls -leOG@ /Users/myuser/Documents

drwx------@ 2 myuser staff datavault 64 17 mar 09:12 DataVaults com.apple.quarantine 19

File attribute datavaultis kept.

Test on ZFS filesystem

4. cp -av ~/Library/Containers/com.apple.mail/Data/DataVaults /Volumes/zDisk/Users/myuser

cp: /Users/myuser/Library/Containers/com.apple.mail/Data/DataVaults: unable to copy extended attributes to /Volumes/zDisk/Users/myuser/DataVaults: Operation not permitted /Users/myuser/Library/Containers/com.apple.mail/Data/DataVaults -> /Volumes/zDisk/Users/myuser/DataVaults

5. ls -leOG@ /Volumes/zDisk/Users/myuser

drwx------@ 2 myuser staff - 2 17 mar 09:12 DataVaults com.apple.quarantine 19

File attribute datavaultis lost.

Side Note cp -a command loses xattr com.apple.rootless, whatever the filesystem , or using sudo. see message ... unable to copy extended attributes ... Even with IP disabled.

• xattr -lv ~/Library/Containers/com.apple.mail/Data/Data*

/Users/myuser/Library/Containers/com.apple.mail/Data/DataVaults: com.apple.quarantine: 0082;00000000;Mail; /Users/myuser/Library/Containers/com.apple.mail/Data/DataVaults: com.apple.rootless: Mail

• xattr -lv /Users/myuser/Documents/Data*

/Users/myuser/Documents/DataVaults/: com.apple.quarantine: 0082;00000000;Mail;`

• xattr -lv /Volumes/zDisk/Users/myuser/Data*

/Volumes/zDisk/Users/myuser/DataVaults: com.apple.quarantine: 0082;00000000;Mail;

[captain-haddock17]

Complementary Test case

retested:

• ZFS xattr=on instead of sa

** same issue** : file flag datavault not present on ZFS filesystem.