rfcs

rfcs copied to clipboard

Extracting configuration file generation out of NixOS service modules in a more reusable way. Based on a discussion during NxCon 2020 Hackday.

This is based on brainstorming during NixCon mainly with @svanderburg . Sander, do you accept to co-sign this as a co-author?

(Also thanks to aanderse jtojnar edolstra 0x4A6F LnL7 for discussion)

Anyone who wants to shepherd this RFC?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/why-isnt-more-of-home-manager-merged-into-nixpkgs/6096/29

I like this RFC, thank you!

@7c6f434c sure! RFC looks great so far. I'm going to read it in detail.

I nominate myself as a shepherd. I don't have practical experience using Nix with other inits, but I am currently working on this.

I’d be interested maybe helping out when it’s in progress. Maybe a flakes-based setup will help realizing this.

https://github.com/svanderburg/nix-processmgmt/ is my favourite prototype so far for abstracting services

I know the example is listed as "a minimalistic silly example" but looking at it I have no idea how the input results in the output

@06kellyjac

agree on nix-processmgmt, but it looks like NixOS is headed towards more reuse of upstream units, and in an RFC aiming to factor something out of NixOS-only I did not want to go too far into things inapplicable to current direction of NixOS.

I think how the input gets converted into the output is an implementation detail? I expect whatever is described before implementation will get even the interface slightly changed once implemented; I do not think specifying internals of the function implementation is a good idea. I guess at first it should be similar to how #42 is prototyped now.

@7c6f434c I wouldn't really want to switch to a new way of defining services, only to then adopt a nix-processmgmt approach and need to do it all over again.

I'm unclear as to whether this RFC is focusing on just configuration files or also the definiton of services, both of which having different implementations across nixos, home-manager, nix-darwin as you've listed. There are a lot of mentions of systemd in the RFC but I think the problem of configuration files and services having different implementations in different projects are two separate but related issues. If the RFC's goal is "Extracting configuration file generation out of NixOS service modules in a more reusable way." then I think the RFC should avoid changing the actual systemd service much/at all and leave that to another RFC.

---

I get not wanting to define implementation details but in the output there are file names like listen.conf and content.conf. where are they defined? what determines that listen.conf is JSON and content.conf is INI? where is it defined that the input port then becomes listenPort for listen.conf?

With #42 I can see how things were before, a proposed structure, and a full example module.

I wouldn't really want to switch to a new way of defining services, only to then adopt a nix-processmgmt approach and need to do it all over again.

Nah, would be out of character for NixOS to adopt init configurability any time soon. On the other hand, #42 is a good idea anyway, and this RFC is more about making the code that would be written anyway available separately from the rest of the things, os not too much of a change of how things are written, just about at what interfaces they are split.

I'm unclear as to whether this RFC is focusing on just configuration files or also the definiton of services, both of which having different implementations across nixos, home-manager, nix-darwin as you've listed.

As summary and detailed design say, this is about configuration files, as there the difference in desired functionality for different targets is pretty small.

There are a lot of mentions of systemd in the RFC but I think the problem of configuration files and services having different implementations in different projects are two separate but related issues.

The mentions about RFC are:

• explanation why the scope does not include complete service generation, only configuration files (systemd unit reuse)
• future work on possible service generation for non-NixOS

If the RFC's goal is "Extracting configuration file generation out of NixOS service modules in a more reusable way." then I think the RFC should avoid changing the actual systemd service much/at all and leave that to another RFC.

… which is why the second mention is future work and not unresolved questions.

I get not wanting to define implementation details but in the output there are file names like listen.conf and content.conf. where are they defined? what determines that listen.conf is JSON and content.conf is INI? where is it defined that the input port then becomes listenPort for listen.conf?

Well, the implementation needs to provide all these facts, which are generally determined by inspecting the program being configured.

With #42 I can see how things were before, a proposed structure, and a full example module.

I am still not sure whether there will be feedback leading to a slight course correction, so I am trying to delay making any kind of implementation prototypes.

As one data point for usecases; I recently started wanting to be able - in nix-on-droid - to start web servers for apps from f-droid that are just frontend clients. My understanding is Termux doesn't support systemd, or something, I imagine standard sysv init scripts will work fine since they are just shell scripts.

...and then I ended up going doing a slight bit more research: There are differences to linux: https://wiki.termux.com/wiki/Differences_from_Linux , probably most importantly no user separation, I'm not sure if you can get fake differring UIDs. Nix-on-droid also uses the proot functionality, which maybe does fake uid0: https://wiki.termux.com/wiki/PRoot

There appears to be a bit of infra for launching service scripts: https://old.reddit.com/r/termux/comments/cs014w/termuxservices_new_package_to_control_daemons/ https://wiki.termux.com/wiki/Termux-services https://wiki.termux.com/wiki/Termux:Boot

A bit of a todo/note to self:

• [ ] if we ever get sufficient functionality to generate services, it would probably be nice to tell termux upstream about it / add it to their wiki

You can observe me naively running into idiosyncrasies of the proot environment in https://github.com/t184256/nix-on-droid/issues/75#issuecomment-682234708 . IIRC the main issues were debugging difficulties and x11 needing patches from termux to handle setuid/setgid exiting with failure because proot refuses to emulate them - so nothing immediately relevant to service generation I think.

Potential use case: https://github.com/ttacon/glorious/issues/49 / https://github.com/numtide/devshell/issues/47

@Profpatsch would you be interested in being a shepherd on this RFC?

Generally this RFC is in need for more shepherds before being able to move onwards.

I nominate myself as a shepherd.

Are you interested in doing shepherd for this RFC, since it might be interested for your projects: @LnL7 @rycee

Nominating @roberth as auther of Arion :)

Nominating @roberth as auther of Arion :)

I should probably mention that Arion doesn't need this,* but may benefit from it, just like other dockerTools users.

Before I'll shepherd anything, I'd like to contribute to the discussion.

As I understand it, this RFC seeks to propose a common interface/convention/abstraction that sits somewhere between pkgs.formats and a "NixOS" module with its platform-specific bits moved to separate module. I'm not convinced that we have enough room between these to two existing concepts to justify another "abstraction level" that we'll all(?) have to maintain.

Have we tried refactoring a module into a common module and platform-specific modules? That can be done today. It's just a matter of "coding to interfaces". That is, importing modules that only declare options. As a real-world example, here's hercules-ci-agent/common.nix, which only depends on nix.* and itself. It works in both NixOS and nix-darwin. Internal options that define the interface between common and platform-specific code are marked as internal, so they don't appear in the documentation.

Alternatively, you can avoid hard config dependencies by checking options ? foo.bar. Interfaces are to be preferred, certainly for the goal of this RFC.

All the integrations listed earlier use the module system anyway, so duplicating all those options as function arguments in a separate file seems like wasted effort. And if an integration isn't based on the module system, it can still invoke the common module because it doesn't need to pull in anything it shouldn't.

A solution like this resolves both drawbacks (although obviously some effort is always required) and I don't see any new drawbacks yet. I'll contrast this solution against the related alternatives that were mentioned:

There also have been many solutions proposed based on a significant rework of the module system.

No rework of the module system required. The nature of the work to be done is similar to the current proposal, but supported by the (existing) module system rather than outside of it.

Implement a complete service abstraction not tied to global system-wide assumptions.

Similar refactoring technique, but we avoid the need for such an abstraction.

If it does come to be, the transition towards such an abstraction is less demanding, because the switch simply increases the scope of the common module. The platform-specific tweaks were already separated out when the common module was introduced, so whatever isn't captured by the service abstraction can simply be kept.

*: Arion supports full systemd-based containers, running NixOS inside the container, so it doesn't have the problem. Nonetheless, init systems (entrypoints) are pluggable in Arion, so a convenient way to define more space-efficient containers/images is a great addition.

Nominating @roberth as auther of Arion :) I should probably mention that Arion doesn't need this,* but may benefit from it, just like other dockerTools users.

I think this is true for most of the use cases. There is the solution of just standartising how the configuration files are handled and then using the partial evaluation of NixOS, which only has the drawbacks of less clear separation (and, also, it avoids the step in the apparently desirable direction of declaring which parts need to see which part of the config).

Before I'll shepherd anything, I'd like to contribute to the discussion.

As I understand it, this RFC seeks to propose a common interface/convention/abstraction that sits somewhere between pkgs.formats and a "NixOS" module with its platform-specific bits moved to separate module. I'm not convinced that we have enough room between these to two existing concepts to justify another "abstraction level" that we'll all(?) have to maintain.

It's not just platform specific bits, it is also all the weird global-scope specific bits.

Have we tried refactoring a module into a common module and platform-specific modules? That can be done today. It's just a matter of "coding to interfaces". That is, importing modules that only declare options. As a real-world example, here's hercules-ci-agent/common.nix, which only depends on nix.* and itself. It works in both NixOS and nix-darwin. Internal options that define the interface between common and platform-specific code are marked as internal, so they don't appear in the documentation.

By now I think that common use of the same modules in NixOS, nix-darwin and home-manager is something that could also happen a few years ago, and it had not, and I assume there are reasons. (Well, there are some reasons like different development cycles and different testing procedures etc.) So I look at what all the solutions do share (Nixpkgs), and I hope to make more code Nixpkgs-like and thus more shareable.

One thing I like in Nixpkgs is that things actually have scoping and so most things only have access to what they need. NixOS modules work on a global level, which is good for some use cases, but I believe it is a good idea to create a path to gradually decouple the code that does not need the global scope from the lobal scope.

Separately, I want a solution that allows organic growth of reuse by the niche parts of the ecosystem, a single clean up helping many exotic platforms (in terms of Platform Tiers RFC).

Alternatively, you can avoid hard config dependencies by checking options ? foo.bar. Interfaces are to be preferred, certainly for the goal of this RFC.

This is not really a solution that makes handling a move of a neighbour to multi-instance modules easier, for example.

All the integrations listed earlier use the module system anyway, so duplicating all those options as function arguments in a separate file seems like wasted effort. And if an integration isn't based on the module system, it can still invoke the common module because it doesn't need to pull in anything it shouldn't.

Module system is a thing that has many uses. The good one is that it is the best typing approach we actually have. The sometimes necessary one is that sometimes you do need to apply a global change to the config based on a policy decision. The annoying one is increasing the coupling. I definitely want the universal parts of NixOS (configuration generation) to be available to setups that only use the module system as a solution for typing, but without global module-based scope.

A solution like this resolves both drawbacks (although obviously some effort is always required) and I don't see any new drawbacks yet. I'll contrast this solution against the related alternatives that were mentioned:

Judging from what happens with Darwin build problems on package bumps, meaningfully Darwin-specific code inside NixOS subtree sounds like a drawback. Sure Hercules-CI has people who use macOS laptops and NixOS servers at once among the core developers, but once the macOS and NixOS user bases are more disjoint, it is a smaller step to have noarch part separate and meaningful platform code separate per platform.

It is true that alternative solutions shoule be expanded (pushed a change).

Implement a complete service abstraction not tied to global system-wide assumptions.

Similar refactoring technique, but we avoid the need for such an abstraction.

Well, we keep many of the global-system properties of a module system in that case.

If it does come to be, the transition towards such an abstraction is less demanding, because the switch simply increases the scope of the common module. The platform-specific tweaks were already separated out when the common module was introduced, so whatever isn't captured by the service abstraction can simply be kept.

with internal options for platform specific implementation details

This misses the point entirely. The dependencies are inverted, with the internal options serving the role of the function output, so the internal options are platform agnostic.

Then I failed to understand the flow completely…

+specific implementation details. Unfortunately, this would force more coupling +of the development cycle for the platform-specific parts. It also does not

So does the function approach.

Not as much, because non-trivial platform-specific environment setup stuff is what gets most of the changes, and hopefully with #42 (which this RFC needs anyway) configuration files per se get to stay closer to the underlying packages in the sense of forcing changes.

+of the development cycle for the platform-specific parts. It also does not +create a clean inspection/override point, and mixes the code with different

Just a matter of calling lib.evalModules, nix repl '<nixpkgs/nixos>', arion repl or whatnot, where the latter variations have the advantage of not having to decipher the logic that produces the inputs. Using options for this is an inspectability improvement over the usual let bindings and such that will be used with the function approach.

There are things that I want to insepect and tweak programmatically. Introducing a functional interface at that point kind of forces that they exist somewhere. Also, then this cut will probably stay in place. Banning let completely in favour of internal options would also solve the problem, but with less chances to introduce stable-ish cut points, and my estimation that such a change has a smaller chance of being adopted.

@@ -123,7 +126,15 @@ widespread use of the module system inside pkgs/. There is also no guarantee that all the configuration files describing interaction of multiple software packages will have a clear choice of reference package.

-There also have been many solutions proposed based on a significant rework of the module system. +A complete or partial merge of the module collections of the major currently +existing module system based code bases, with internal options for platform +specific implementation details. Unfortunately, this would force more coupling +of the development cycle for the platform-specific parts. It also does not +create a clean inspection/override point, and mixes the code with different +platform requirements for testing.

Again, the common module isn't platform specific and can stand alone. It's really just like a function, except you get to reuse the option declarations for its inputs.

The two approaches are isomorphic if it wasn't for the fact that going back to a simple function loses the option metadata and checking.

The function is already specified to come with the option metadata, though, that can be then imported into the options specification.

It is forced not to care where in the namespace it will get imported, though.

inputs <-> normal options outputs <-> internal options multiple invocations <-> submodules function without free variables <-> module that only defines self-declared options function with small closure <-> module that doesn't import much

+generation code closer to the Nixpkgs model with scope/visibility rules. At +the same time we aim to avoid or minimise code in the NixOS subtree that +cannot be tested on NixOS.

This is not a goal but an implementation choice.

Judging from observations of Nixpkgs workflows, it is a useful goal to have…

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/why-isnt-more-of-home-manager-merged-into-nixpkgs/6096/33

Reading & asking, I wonder, by virtue of a faint suggestion, if on a high level I'm correctly connecting the dots:

Is there a treasure trove in configuration know-how to be lifted from home-manager out of its silo by mediation of the principles proposed in this RFC?

Remember that home-manager is implemented entirely in nix. Nothing formally prevents any part of it from being absorbed into nixpkgs besides politics and sentiment.

I believe the previous discussion to merge home-manager into nixpkgs was without any deduplication plans, so all of the coordination overhead, none of the technical benefits. If the configuration database behaves closer to package database and doesn't care what's the structure above it — well, home-manager seems to work fine with upstream Nixpkgs.

@roberth hmmm, actually, do you know of any support for module import as a subtree, i.e. importing a standalone module with options like «enable» and «port» to be at path «services.newssh», i.e. with options «services.newssh.enable» and «services.newssh.port»?

I agree that it would be cleaner than a function with a module signature for type checking on the side.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixflk-template-repo-for-nixos-configurations-using-flakes/5325/13

@7c6f434c

@roberth hmmm, actually, do you know of any support for module import as a subtree, i.e. importing a standalone module with options like «enable» and «port» to be at path «services.newssh», i.e. with options «services.newssh.enable» and «services.newssh.port»?

Yes.

# this wouldn't be a let but separate files
let

  inherit (import <nixpkgs/lib>) evalModules types mkOption optionalString;

  pureModule = { config, ... }: {
    options = {
      # inputs
      port = mkOption { default = 22; /* etcetera */ };

      # outputs
      configText = mkOption { internal = true; default = ''
        Port ${toString config.port}
      ''; };
    };
  };

  liftedModule = {
    options.services.ssh = mkOption {
      type = types.submodule pureModule;
    };
    config.services.ssh = {};
  };

  nixosBase = {
    # Let's pretend :)
    options.system.build.activationScript = mkOption { type = types.lines; };
  };

  nixosSshModule = { config, ... }: {
    options.services.ssh = mkOption {
      type = types.submodule {
        options.socketActivated = mkOption { default = true; /* etcetera */ };
      };
    };
    config.system.build.activationScript = ''
      Start openssh ${optionalString config.services.ssh.socketActivated "via socket"} with config file ${config.services.ssh.configText}
    '';
  };

  userConfiguration = {
    services.ssh.port = 2222;
  };

in
  evalModules { modules = [liftedModule nixosBase nixosSshModule userConfiguration]; }

$ nix repl subtree.nix 
Welcome to Nix version 2.4pre20201201_5a6ddb3. Type :? for help.

Loading 'subtree.nix'...
Added 3 variables.

nix-repl> config.system.build.activationScript
"Start openssh via socket with config file Port 2222\n\n"

We might benefit frome syntactic sugar for liftedModule, so that nixosSshModule can import it all by itself, but let's not jump to conclusions until we know how the usage of submodule merging develops.

There are things that I want to insepect and tweak programmatically. Introducing a functional interface at that point kind of forces that they exist somewhere. Also, then this cut will probably stay in place.

I've updated the example in my previous comment to include the pure function idea. pureModule can be called independently using evalModules.

In this example I had to modify the file, but normally those modules would be separate files, so you could just reference the module you need.

# let ... in
  evalModules { modules = [ pureModule ]; }

$ nix repl subtree.nix 
Welcome to Nix version 2.4pre20201201_5a6ddb3. Type :? for help.

Loading 'subtree.nix'...
Added 3 variables.

nix-repl> config.configText
"Port 22\n"

Banning let completely in favour of internal options would also solve the problem, but with less chances to introduce stable-ish cut points, and my estimation that such a change has a smaller chance of being adopted.

I agree. Banning let entirely doesn't make sense. I should have clarified which let binding: the one with the configuration text or config file derivation. (OT: text is preferable when dealing with secrets)

[If?] the function is already specified to come with the option metadata, though, that can be then imported into the options specification. It is forced not to care where in the namespace it will get imported, though.

Exactly. If the module file sits next to the Nixpkgs package definition, I think expectations are managed pretty well. We can also have a unit test for the config file generation, which will enforce the purity aka "not caring where in the namespace it will get imported". The unit test can be added to passthru.tests, but I expect that if you try to pass the settings module via passthru, you run into a strictness issue. We can still reference the file by path though.

Interesting, thanks a lot! Indeed that lifting sounds nice and with a minimal test could guarantee the same pure-function properties with smoother integration than separate module definitions alongside a function.

I agree that settins / configText / configFile should be visible/overridable simultaneously (in the submodule approach it makes more sense to specify that).

liftedModule does sound like something I kind of hope to have autogenerated inside the system-level module… Not sure if it is worth trying to add liftingImports into the modules system or there is a better way.