Persisting `/var/lib/private/servicename` creates the parent directory with wrong permissions `0755`

[oddlama]

I've persisted some service directories that use DynamicUser=true, which usually have their state directories in /var/lib/private/servicename. Since some month I noticed that some of my services fail to start because /var/lib/private is set to 0755 instead of 0700. I'm pretty sure that this wasn't the case a year ago but I noticed that impermanence seems to be responsible for this:

/nix/store/xj5zys52xvdc9wzsrhwlcbcwvnn3bkiq-impermanence-run-create-directories:

# ...
/nix/store/r4cs4lnkwjjd7yx29245cz8mvpc2pwbh-impermanence-create-directories /persist /var/lib/private root root 0755 ''
/nix/store/r4cs4lnkwjjd7yx29245cz8mvpc2pwbh-impermanence-create-directories /persist /var/lib/private/esphome root root 0700 ''
# ...

How can I enforce this directory to be created 0700? I've tried using systemd-tmpfiles to do it after the fact, but alas this didn't help.

[diogotcorreia]

Not sure if this is the intended solution, but I've had success setting defaultPerms.mode to 0700, as such:

{
  dir = "/var/lib/private/esphome";
  mode = "0700";
  defaultPerms.mode = "0700";
}

See https://github.com/diogotcorreia/dotfiles/commit/bf63e99cc7a477395c9ae5bcb8108c69afcdfcf2

However, this appears to set 0700 for every parent, including /persist, which is definitely undesirable. Fortunately (and important if you are fixing permissions now), impermanence ignores these permissions if the directory already exists under /persist, so you have to change them manually for /persist/var/lib/private after the fix.

---

Also, I haven't tested, but the intended solution might just be to define

{
  dir = "/var/lib/private";
  mode = "0700";
}

in addition to the child directories. If you can, give that a go and report back

[oddlama]

Also, I haven't tested, but the intended solution might just be to define

{ dir = "/var/lib/private"; mode = "0700"; }

in addition to the child directories. If you can, give that a go and report back

This will definitely work around the problem, but it also will persist the whole /var/lib/private directory. That kind of defeats the whole purpose of adding the child directories in the first place and I explicitly don't want to persist some other directories in there.

I think a "correct" solution would be to allow users to define modes for certain parents without forcing those parents to become persisted.

[diogotcorreia]

This will definitely work around the problem, but it also will persist the whole /var/lib/private directory.

Didn't think of that, that's definitely a problem as well.

I've been thinking about how the directories are generated and it's important to note that the order matters. For example, my snippet above with defaultPerms.mode generates the following activation script:

/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories '' /nix root root 0755 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories '' /nix root root 0700 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories '' /nix/persist root root 0755 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories '' /nix/persist root root 0700 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var root root 0755 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var/lib root root 0755 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var root root 0700 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var/lib root root 0700 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var/lib/private root root 0700 ''
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /etc root root 0755 ''
# ...
/nix/store/l2r1dk6k79bgxn4czvhc43kmgkr1zk0m-impermanence-create-directories /nix/persist /var/lib/private/wastebin root root 0700 ''
# ...

In a clean install, the directory gets the mode of the first command, so /nix, /nix/persist, /var and /var/lib would get 0755, while /var/lib/private would get 0700, which is exactly what we want! However, this is just due to the other stuff I already have declared, because if I didn't have anything else persisted, all of those directories would get 0700 as well.

I think a "correct" solution would be to allow users to define modes for certain parents without forcing those parents to become persisted.

All of this is to say that you're right and we need a proper solution, there doesn't seem to exist any option at the moment to allow us to manage parents' permissions properly. Also, defaultPerms is marked as internal = true;, so we probably should not be using it like this 🙃

Relevant code: https://github.com/nix-community/impermanence/blob/4b3e914cdf97a5b536a889e939fb2fd2b043a170/nixos.nix#L669

[tomfitzhenry]

I too noticed this situation started a few months ago, with /var/lib/private/kea. Maybe systemd previously eagerly created /var/lib/private/ on startup, but now defers it to service startup?

I think a "correct" solution would be to allow users to define modes for certain parents without forcing those parents to become persisted.

This seems to work:

systemd.tmpfiles.rules = [
  "d /var/lib/private 0700 root root"
];

Perhaps impermanence can define this for well-known directories such as /var/lib/private?

[oddlama]

This seems to work:

systemd.tmpfiles.rules = [
  "d /var/lib/private 0700 root root"
];

Perhaps impermanence can define this for well-known directories such as /var/lib/private?

This does not suffice for me, when switching to a new configuration the impermanence script will run again and modify the existing directory again to the wrong permissions values

[codyps]

I've ended up tweaking the activation to ensure the /persist/var/lib/private (replace /persist with whatever your persistent storage is) has the right permissions before the impermanence activation copies those permissions to the /var/lib/private dir.

  system.activationScripts."createPersistentStorageDirs".deps = [ "var-lib-private-permissions" "users" "groups" ];
  system.activationScripts = {
    "var-lib-private-permissions" = {
      deps = [ "specialfs" ];
      text = ''
        mkdir -p /persist/var/lib/private
        chmod 0700 /persist/var/lib/private
      '';
    };
  };

Should make things would if /persist/var/lib/private is deleted and on first creation. Minimal validation though, and poking at activationScripts.*.deps feels funny.