how to enforce a dependency on sops-nix for systemd services ?

[teto]

first of all apologies since it's not really a sops-nix issue but the issue might be worth documenting and I am sure you have an answer somehow. I have my deployments failing because of a racing issue I believe, e.g., some services try to start before the secrets are available ? For instance

× gitlab-runner.service - Gitlab Runner
     Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Fri 2022-09-02 09:01:20 UTC; 40ms ago
       Docs: https://docs.gitlab.com/runner/
    Process: 2402 ExecStartPre=/nix/store/v8cpddy7djagbcyilbig47sks79xmymk-gitlab-runner-configure/bin/gitlab-runner-configure (code=exited, status=1/FAILURE)
         IP: 5.5K in, 1.8K out
        CPU: 534ms

Sep 02 09:01:19 aws gitlab-runner-configure[2441]: cat: /run/secrets/gitlab/registrationToken: No such file or directory
Sep 02 09:01:19 aws gitlab-runner-configure[2440]: Runtime platform                                    arch=amd64 os=linux pid=2440 revision=v15.2.1 version=15.2.1

the most annoying being sshd since i get locked out of the machine.

I tried to fix the dependency between sshd and sops-nix like this

  # has to be deployed on
  systemd.services.sshd = {
    requires = [ "run-secrets.d.mount" ];
    after = [ "run-secrets.d.mount" ];
  };

but it doesn't to fix it, e.g., I still see the previous error.

[teto]

Also sometimes I forget to upload the age key on the server so sops fails which triggers the error in my previous message

Activation script snippet 'setupSecrets' failed (1)
/nix/store/x3gv1pgkcdp423cnix9qh3s9fmvb2rcz-sops-install-secrets-0.0.1/bin/sops-install-secrets: Cannot read keyfile '/var/lib/sops-nix/key.txt': open /var/lib/sops-nix/key.txt: no such file or directory
reloading user units for root...

I suppose there is no way to cancel the nixos-rebuild in activationScript but could the failure of sops prevents the service of sshd to be restarted.

[hmenke]

How about something like this?

{
  systemd.services."my-service" = {
    enable = true;
    unitConfig = {
      # Service activation will be skipped if this file doesn't exist
      ConditionPathExists = "/run/secrets/my-secret";
    };
  };
  systemd.paths."my-secret" = {
    enable = true;
    pathConfig = {
      # As soon as the specified file is created start the service
      PathExists = "/run/secrets/my-secret";
      Unit = "my-service.service";
    };
  };
}

Detailed documentation: https://www.freedesktop.org/software/systemd/man/systemd.unit.html https://www.freedesktop.org/software/systemd/man/systemd.path.html

[Mic92]

I think this might still trigger a service restart never the less.

[nipsy]

Since I ran across @hmenke's comment above as a potential solution to my sopx-nix problem regarding keeping parts of my nftables configuration private in my repo, I figured I'd at least share what I learned using that comment as a starting point.

As written above, current NixOS at least doesn't actually start any of those related services. Per the NixOS options documentation for enable, that doesn't actually start these services under systemd. Additionally, if we use the same name for the path and the service, we can skip specifying the Unit line altogether. Which ends up looking like:

  systemd.services."nftables-extra" = {
    description = "nftables extra firewall rules";
    script = ''
      ${pkgs.nftables}/bin/nft -f ${config.sops.secrets."nftables/ssh".path}
    '';
    serviceConfig = {
      RemainAfterExit = true;
      Type = "oneshot";
    };
    unitConfig = {
      ConditionPathExists = config.sops.secrets."nftables/ssh".path;
    };
    wantedBy = [ "multi-user.target" ];
  };
  systemd.paths."nftables-extra" = {
    pathConfig = {
      PathExists = config.sops.secrets."nftables/ssh".path;
    };
    wantedBy = [ "multi-user.target" ];
  };

I'm not sure the RemainAfterExit or Type are strictly necessary, but I followed the service definition as used by NixOS for nftables itself at https://github.com/NixOS/nixpkgs/blob/nixos-unstable/nixos/modules/services/networking/nftables.nix. And specifying the wantedBy lines as recommended in the options documentation for enable does, in fact, start both services successfully. Any number of additional rules can be defined in the same service by simply adding additional invocations of 'nft -f' pointing at the proper secret paths (if needed) to the script string above.

In case anyone else should stumble upon this and want to do something similar, since I wasn't able to do this any other way (I think due to the known limitation of secrets not being available until activation), this additional table I'm adding here complements the built-in NixOS firewall table but takes priority over handling SSH port traffic (due to the "filter - 1" priority), regardless of the settings of services.openssh.openFirewall, all while still allowing everything else to get handled by the default nixos-fw inet table configured by most of the usual NixOS related options (due to the "policy accept" which is the default but I still prefer to be explicit about it to make the rule set easier to follow):

table inet ssh;
flush table inet ssh;
table inet ssh {

        define guarded_ports = {ssh}

        define port1 = 12345
        define port2 = 23456
        define port3 = 34567
        define port4 = 45678

        set clients_ipv4 {
                type ipv4_addr
                flags timeout
        }

        set clients_ipv6 {
                type ipv6_addr
                flags timeout
        }

        set candidates_ipv4 {
                type ipv4_addr . inet_service
                flags timeout
        }

        set candidates_ipv6 {
                type ipv6_addr . inet_service
                flags timeout
        }

        chain input {
                type filter hook input priority filter - 1; policy accept;

                tcp dport $port1 add @candidates_ipv4 {ip  saddr . $port2 timeout 2s}
                tcp dport $port1 add @candidates_ipv6 {ip6 saddr . $port2 timeout 2s}

                tcp dport $port2 ip  saddr . tcp dport @candidates_ipv4 add @candidates_ipv4 {ip  saddr . $port3 timeout 2s}
                tcp dport $port2 ip6 saddr . tcp dport @candidates_ipv6 add @candidates_ipv6 {ip6 saddr . $port3 timeout 2s}

                tcp dport $port3 ip  saddr . tcp dport @candidates_ipv4 add @candidates_ipv4 {ip  saddr . $port4 timeout 2s}
                tcp dport $port3 ip6 saddr . tcp dport @candidates_ipv6 add @candidates_ipv6 {ip6 saddr . $port4 timeout 2s}

                tcp dport $port4 ip  saddr . tcp dport @candidates_ipv4 update @clients_ipv4 {ip  saddr timeout 10s} log prefix "Successful portknock: "
                tcp dport $port4 ip6 saddr . tcp dport @candidates_ipv6 update @clients_ipv6 {ip6 saddr timeout 10s} log prefix "Successful portknock: "

                tcp dport $guarded_ports ip  saddr @clients_ipv4 counter accept
                tcp dport $guarded_ports ip6 saddr @clients_ipv6 counter accept
                tcp dport $guarded_ports ct state established,related counter accept

                tcp dport $guarded_ports counter reject with tcp reset
        }
}