Access secret from systemd service with DynamicUser=true

[matrss]

I use a nixos service which utilizes systemd's DynamicUser feature (tiddlywiki). I am unsure how to make a secret accessible only to this service. The usual approach of setting the owner of the secret to the user the service runs as does not work, as there is no static user created.

For now I have added the keys group as a supplementary group to the service and made the secret accessible to it, but this would mean that every service configured this way would have access to all the secrets available, which I would rather like to avoid.

[matrss]

As usual, you find a solution right after explaining the problem...

The solution might be systemd's LoadCredential. In particular, this comment helped me out: https://github.com/NixOS/nixpkgs/issues/102397#issuecomment-853472016.

As an example, my tiddlywiki config now looks like this:

sops.secrets."tiddlywiki/credentials" = { };

services.tiddlywiki.enable = true;
services.tiddlywiki.listenOptions = {
  readers = "(anon)";
  writers = "(authenticated)";
  admin = "matrss";
  credentials = "/run/credentials/tiddlywiki.service/credentials";
  host = "127.0.0.1";
  port = "8080";
};

systemd.services.tiddlywiki.serviceConfig.LoadCredential =
  "credentials:${config.sops.secrets."tiddlywiki/credentials".path}";

Of course, this might be simplified and integrated in some way or another...

[Mic92]

A different alternative is to set the User parameter in serviceConfig. Systemd will than provide the user -> uid resolution through nss-systemd. Here an example: https://github.com/Mic92/dotfiles/blob/4580668bca44d651f1baf4c8280336b16fdd06b8/nixos/eva/modules/prometheus/default.nix#L289 Than you can set .owner field as usual.

[matrss]

Oh, that's a lot more elegant, as it does not make assumptions about the location of the systemd credentials directory. Thanks!

Should this issue be closed then, or do you see some way in which sops-nix could be improved here?

[matrss]

Actually, this does not the seem to work for me.

With the following configuration:

sops.secrets."tiddlywiki/credentials" = {
  owner = "tiddlywiki";
};

services.tiddlywiki.enable = true;
services.tiddlywiki.listenOptions = {
  readers = "(anon)";
  writers = "(authenticated)";
  admin = "matrss";
  credentials = config.sops.secrets."tiddlywiki/credentials".path;
  host = "127.0.0.1";
  port = "8080";
};

systemd.services.tiddlywiki.serviceConfig.User = "tiddlywiki";

I get the error:

error: attribute 'tiddlywiki' missing

       at /nix/store/24yq1h01mmf9yp34vgc1prd9582azfzd-source/modules/sops/default.nix:66:19:

           65|         type = types.str;
           66|         default = users.${config.owner}.group;
             |                   ^
           67|         defaultText = literalExpression "config.users.users.\${owner}.group";

If I also specify the group for the secret, then I get an error at activation:

/nix/store/8ryvmsvzdz2fb2yw26lkrdv4fr3116d0-sops-install-secrets-0.0.1/bin/sops-install-secrets: Manifest is not valid: Failed to lookup user 'tiddlywiki': user: unknown user tiddlywiki
Activation script snippet 'setupSecrets' failed (1)

[Mic92]

If you specify a group, than you need to do the same in the systemd service:

systemd.services.tiddlywiki.serviceConfig.User = "tiddlywiki";
systemd.services.tiddlywiki.serviceConfig.Group = "tiddlywiki";

[Mic92]

If you get uid errors during upgrade, is it possible that nscd was restarted when you got this issue?

[matrss]

If you specify a group, than you need to do the same in the systemd service:

systemd.services.tiddlywiki.serviceConfig.User = "tiddlywiki";
systemd.services.tiddlywiki.serviceConfig.Group = "tiddlywiki";

Yes, that is what I did. Unfortunately it does not work for me.

If you get uid errors during upgrade, is it possible that nscd was restarted when you got this issue?

I do not think so. The full activation output when I am testing this is

stopping the following units: tiddlywiki.service
activating the configuration...
setting up /etc...
/nix/store/yx24pwdnka50lcfi1h5l7y64sajxj7xi-sops-install-secrets-0.0.1/bin/sops-install-secrets: Manifest is not valid: Failed to lookup user 'tiddlywiki': user: unknown user tiddlywiki
Activation script snippet 'setupSecrets' failed (1)
reloading user units for root...
reloading user units for matrss...
setting up tmpfiles
starting the following units: tiddlywiki.service

(I am using deploy-rs for this server and skipped the output before and after the activation step)

[ritave]

@Mic92 I'm also having the same problem, I have set both the user and the group.

let goerliUser = "goerli"; in
{
  sops.secrets."goerli.jwt" = {
    owner = goerliUser;
    group = goerliUser;
  };
  # Note that in nixpkgs, that service has DynamicUser = true
  systemd.services.geth-goerli.serviceConfig = {
    User = goerliUser;
    Group = goerliUser;
  };
}

and it errors out with

building the system configuration...
stopping the following units: geth-goerli.service
activating the configuration...
setting up /etc...
/nix/store/4q0v00wl26jjvl2i5zqcp6kinglnkkk3-sops-install-secrets-0.0.1/bin/sops-install-secrets: Manifest is not valid: Failed to lookup user 'goerli': user: unknown user goerli
Activation script snippet 'setupSecrets' failed (1)
reloading user units for ritave...
setting up tmpfiles
starting the following units: geth-goerli.service
warning: error(s) occurred while switching to the new configuration

[Mic92]

@Mic92 I'm also having the same problem, I have set both the user and the group.

let goerliUser = "goerli"; in
{
  sops.secrets."goerli.jwt" = {
    owner = goerliUser;
    group = goerliUser;
  };
  # Note that in nixpkgs, that service has DynamicUser = true
  systemd.services.geth-goerli.serviceConfig = {
    User = goerliUser;
    Group = goerliUser;
  };
}

and it errors out with

building the system configuration...
stopping the following units: geth-goerli.service
activating the configuration...
setting up /etc...
/nix/store/4q0v00wl26jjvl2i5zqcp6kinglnkkk3-sops-install-secrets-0.0.1/bin/sops-install-secrets: Manifest is not valid: Failed to lookup user 'goerli': user: unknown user goerli
Activation script snippet 'setupSecrets' failed (1)
reloading user units for ritave...
setting up tmpfiles
starting the following units: geth-goerli.service
warning: error(s) occurred while switching to the new configuration

There is a known race condition if nscd is stopped in the second systemd wants resolve this user. I think there has been some fixes added in nixpkgs-unstable and the upcoming release to resolve this. You might also want to try services.nscd.enableNsncd = true; which is an alternative implementation of the unmaintained nscd.

Update mhm, since this happens in activation phase, I am actually not so sure if this is the same issue. Do these issues go away on the second deploy?

[Mic92]

If you can I would try to make use of systemd's LoadCredential as it solves these types of problems.

[ritave]

@Mic92 I confirm the issues go away on the second deploy. I'm working from nixpkgs#master branch, so any fixes that are there, are not fixing the problem yet.

I'll try to go with the credentials route

[Mic92]

@Mic92 I confirm the issues go away on the second deploy. I'm working from nixpkgs#master branch, so any fixes that are there, are not fixing the problem yet.

I'll try to go with the credentials route

Fixing the issue is in sops-nix interest however need to happen in nixos, I am afraid. The only workaround I could think of just now is to retry resolving users if they are not found...

[chreekat]

Since a dynamic user doesn't exist in any form before systemd (enables? loads?) a module that declares one, how is sops-nix supposed to check if the user exists? I don't see how this could be fixed in nixos :)

[mikepurvis]

It would be great to have an example of how LoadCredential may be used for this. I'm a novice to both NixOS and systemd, and it looks like it would be a bit of a garden path:

• in sops-nix config, set the secret to be a file in /run/secrets/foo that's readable only by root.
• in serviceConfig.LoadCredential, have systemd (as root) read that file and pass its contents into the unit's environment under name foo.
• set the unit's application-level config to look for its secret in ${CREDENTIALS_DIRECTORY}/foo, however that's a special ExecStart substitution, not a real envvar— if you need to load the path from an envvar, you have to pass it in with Environment=FOOPATH=%d/foo

---

Edit: Oh wait, maybe Harmonia's module is already a great example of this in action— as long as you set the signKeyPath option, it correctly handles the LoadCredential side of things:

https://github.com/nix-community/harmonia/blob/b5d77f05256529edbaf574a478d16a8570826789/module.nix#L44-L68

[pmario]

@matrss Hi, Did you ever get your NixOS tiddlywiki configuration up and running in a reasonable securely manner?

I would be interested in a working configuration. There is a forum post at Talk-TiddlyWiki, which imo would be the best way to let us know. Thx in advance.

[matrss]

@pmario I am currently running tiddlywiki with the nixpkgs' provided service and requiring authentication for both readers and writers. You can see most of my configuration regarding that in tiddlywiki.nix. It is still using the LoadCredential approach from https://github.com/Mic92/sops-nix/issues/198#issuecomment-1166638925. There is some nginx configuration in nginx.nix as well, which also applies to serving TW5 (the entire repo contains the configuration for all my NixOS systems; it is using flakes. Feel free to explore, but it might be a bit messy and rather un-documented in many places).