Mode used in Node/Tree is specific to go and might change in future go versions?

[aawsome]

Sorry for not using the template. This issue is not a concrete bug, but something I observed when developing rustic

The point is that in internal/restic/node.go the following mask is used for the modes which are saved in the trees:

mask := os.ModePerm | os.ModeType | os.ModeSetuid | os.ModeSetgid | os.ModeSticky

The docu says:

The defined file mode bits are the most significant bits of the FileMode. The nine least-significant bits are the standard Unix rwxrwxrwx permissions. The values of these bits should be considered part of the public API and may be used in wire protocols or disk representations: they must not be changed, although new bits might be added.

With other words: restic uses part of the non-public API and saves this information in the on-disc representation.

For rustic, this has the effect that I wasn't able to easily reproduce the file modes (even though the last nine bits were always correct). The effect is that when using rustic on an restic repository and backup an completely unchanged path, it correctly detects the parent snapshot and also correctly identifies that no file has been changed - but most or all trees are newly written as the modes changed. Hence, afterwards, two versions of the trees are saved which means double storage used for trees.

I don't know how stable this non-public API in go is, but if it will happen to change, users will experience the same when using restic compiled with different go versions on the same repository.

Actually I don't know a good way to solve this. My impulse would be to only save the mode umasked by 0777, but this will simply enforce the unwanted behavior I depicted above right now...

[fd0]

As far as I understand the docs you linked all bits (not only the lowest 9 bits) are part of the public API of the Go standard library, so I don't expect them to change. Especially, the bit definitions are not dependent on a file system or architecture.

I'll think about what to do. For example, we could add the meaning of the bits to the restic spec and add tests that fail should the bits ever change. In that case when can then write a conversion function or so.

[aawsome]

Actually, re-reading it you might be right that all bits and not only the nine ones are the public API.

Then, maybe the best way would be to add this to the specs and I'll try to reimplement it in rustic...

[aawsome]

I'm still struggling with this very restic/golang-specific way of saving file modes. There is the (POSIX?) standard described in the man page inode(7) which seems to be used in C/C++/Python/Rust. For me, this would be the ideal choice for persisting file modes.

Maybe golang tried to make this platform-independent, but IMO that approach is doomed anyway as there will be always platforms which cannot be covered.

Anyway, I re-implemented this weird golang modes in rustic and will soon propose a PR to be more precise in the design docu. (actually there are some other minor gotchas in the tree format you stumble over when trying to reimplement only based on the design document :wink: )