Improve --exclude-cloud-files to ignore iCloud-only files on macOS

[hashier]

Output of restic version

restic 0.18.0 compiled with go1.24.1 on darwin/arm64

What should restic do differently? Which functionality do you think we should add?

Improve/extend --exclude-cloud-files to also work on macOS/darwin with iCloud files. Prior work that implemented this for windows:

• https://github.com/restic/restic/issues/3697
• https://github.com/restic/restic/pull/4990

What are you trying to do? What problem would this solve?

Be able to ignore iCloud files that are only stored in the cloud and not locally. (A reason to write/learn some go code and provide something to restic)

Did restic help you today? Did it make you happy in any way?

Every day! Recently moved to a cron job (launchd agent) for restic. Had it for 3 years running manually but thought this should be automated. Now my system gets daily backups instead of just every 2 weeks when I remember to run restic.

Additional info

~~I wrote a small POC for this change. If no one is already working on it I would try to follow the PR that implemented this feature on Windows~~

PR that implements this feature

https://github.com/restic/restic/pull/5370

POC Code

package main

import (
	"fmt"
	"os"
	"syscall"

	"golang.org/x/sys/unix"
)

func main() {
	isCloudFile("a")
	isCloudFile("b")
}

func isCloudFile(filePath string) {
	fileInfo, _ := os.Stat(filePath)
	v, ok := fileInfo.Sys().(*syscall.Stat_t)
	if !ok {
		fmt.Println("Can't get flags of file", filePath)
	}
	const mask uint32 = unix.SF_DATALESS // 0x40000000
	if v.Flags&mask == mask {
		fmt.Println(filePath, "is cloud file")
	} else {
		fmt.Println(filePath, "is normal file")
	}
}

[MichaelEischer]

I wrote a small POC for this change. If no one is already working on it I would try to follow the PR that implemented this feature on Windows

I'm not aware of other issues in that regard, so feel free to open a PR.

[konidev20]

Some references about the topic:

1. TN3150: Getting ready for dataless files
2. macOS Sonoma has changed iCloud Drive radically
3. https://news.ycombinator.com/item?id=40769838

Check if a file is dataless and then only access it in a safe context. To do the check, call stat or getattrlist and examine if SF_DATALESS is present in stat.st_flags. Be aware that stat and getattrlist both trigger the materialization of any intermediate folders in the file’s path, if they themselves are dataless.

From [1]

[konidev20]

I noticed one thing @hashier, the code snippet you shared works only for files. I tried creating a folder on iCloud Drive in MacOS Sequoia 15.x. added some files in to the folder and then selected Remove Download. The file inside of the folder was correctly detected as a cloud file, but not the folder.

Now, if there are thousands of files in iCloud Drive, ideally we should not enter the directory itself right? I am not sure how the Windows version works for OneDrive and Sharepoint files.

Please let me know if I am missing something.

[hashier]

@konidev20 Thanks for the shared links

For the folder "issue". I saw the same. Even though the docs mention that folders should be able to be cloud-only, I never managed to create one:

$ stat -s cloud-only/
st_dev=16777230 st_ino=6238970 st_mode=040755 st_nlink=4 st_uid=501 st_gid=20 st_rdev=0 st_size=128 st_atime=1744549850 st_mtime=1744549845 st_ctime=1744549846 st_birthtime=1744549830 st_blksize=4096 st_blocks=0 st_flags=0

note: st_flags=0 (finder shows this folder as cloud-only, but not stat)

[hashier]

PR is available here: https://github.com/restic/restic/pull/5370 would love if someone could review/merge it (: