rfcs
rfcs copied to clipboard
NixOS
[RFC 0163] Portable Service Layer
This RFC proposes a generic, portable service layer that makes it possible to manage services on top of (in theory) any operating system that is capable of running the Nix package manager.
This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/rfc-0160-portable-service-layer/33886/1
PSA: RFC numbers should match the PR number, so this is RFC 163. I updated the title, but make sure to also update the file path.
PSA: RFC numbers should match the PR number, so this is RFC 163. I updated the title, but make sure to also update the file path.
Ah, I wasn't aware of this. I just updated the file
The title of this PR should probably be "[RFC 163] Portable Service Layer" or something along those lines, not "Initial commit" :^)
The title of this PR should probably be "[RFC 163] Portable Service Layer" or something along those lines, not "Initial commit" :^)
Indeed. I just updated the commit and PR title
Does this RFC cover dynamic service orchestration or is that our of scope?
If I, as a maintainer for some NixOS module, do not care for other init systems but systemd, I must be able to write the module in such a way that it is only compatible with systemd.
A mandatory non-portable module? Why?
I must still be able to use e.g. systemd's ExecStart with its exact semantics
You want access to systemd the same way the keyword asm provides access to the assembler?
Also, how portable is that systemd interface?
This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/duplication-home-manager-and-nixos-modules/30862/28
Does this RFC cover dynamic service orchestration or is that our of scope?
What do you exactly mean with "dynamic"? Is it about deploying processes that are not part of a NixOS system profile?
Does this RFC cover dynamic service orchestration or is that our of scope?
What do you exactly mean with "dynamic"? Is it about deploying processes that are not part of a NixOS system profile?
I like to think of first-class orchestrators. One can imagine this system as an orchestrator. If this system deploys an orchestrator as a service, then that results in some level of dynamic orchestration.
To add a bit of other "prior art", I'd like to mention a project I've been working on recently, which extends the drv-parts(Now part of dream2nix) to define a generic service layer with the module system which can be rendered into configurations for multiple process managers. The project is still in development, but the core module api is pretty much finnished. It allows defining multiple services together in a straightforward way. See this example:
deployments.hello.services = {
hello = {
imports = [
../packages/hello.nix #uses the hello package defined in a drv-parts/dream2nix module
];
service = {
args = [
"-g"
''"Hello there, $GENERAL!"''
];
env.GENERAL = "General Kenobi";
};
};
other-service = {
imports = [
../packages/hello.nix
];
service.args = [
"-g"
''"Hello from the other service!"''
];
};
};
Note that this setup could also easily be used without drv-parts/dream2nix, by requiring services to set a "mainProgram" option instead.
This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/rfc-0163-portable-service-layer/33886/3
This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/what-is-the-equivalent-of-configuration-nix-for-just-the-nix-package-manager/34070/2
Excellent. Immediate benefits aside this kind of abstraction would be immensely helpful for OS research.
I'm wondering how far the scope of "services" extends above the system-level. Could graphical desktop applications be included? That's a big scope-creep but I imagine there is more incentive to write fresh desktop services than to migrate NixOS services that already work fine where they are.
I run Synit as a second-level init system and I can't be bothered to rewrite the systemd services from NixOS, but I do write services from-scratch for my desktop environment.
Could graphical desktop applications be included?
It looks very natural, since Home-Manager has support for X11 window managers.
I have a plan to assimilate Home Manager to Nixpkgs repo. Can your RFC help this transition, since it would provide a portable service layer to HM too?
If I, as a maintainer for some NixOS module, do not care for other init systems but systemd, I must be able to write the module in such a way that it is only compatible with systemd.
A mandatory non-portable module? Why?
I assume the point is «I want some weird kind of dependency that I barely managed to figure out with systemd and don't know to express in anything more generic»
A mandatory non-portable module? Why?
As I elaborated in my initial comment, this is in order to preserve the ability to use every feature of systemd with its exact semantics as we currently can.
I'm not saying this should be the default or even desirable but it should be an option; a valid one at that.
You want access to systemd the same way the keyword
asmprovides access to the assembler?
Yes. In most cases you shouldn't need it and I too would prefer to use more abstract interfaces but we must also preserve the ability to use the "bare metal" access.
Niche process manager support cannot come at significant burden for everyone else.
Also, how portable is that systemd interface?
What do you mean by "portable"? Systemd only supports Linux currently.
As a NixOS user and maintainer, I should not be forced to care about non-systemd+linux platforms. I should have the option to but not the obligation.
I want some weird kind of dependency that I barely managed to figure out with systemd and don't know to express in anything more generic
The problem isn't just knowledge, it's having to care; if I cared, I'd know.
I don't want to have to care about the specifics of 6 different other service managers that were all replaced by a (IMO) superior option that is in use today (status quo).
Also, forcing the use of generic interfaces must come at the cost of a loss in configurability as the generic interface cannot be more than the intersection of all process managers' features. If that generic interface was the only option, there'd be no reason to use systemd anymore because none of its desirable features are supported by the legacy process managers it replaced.
Therefore, as a NixOS user and maintainer, I must be able to use non-generic/specific interfaces if I need to.
(That should ideally work the other way aswell; if some process manager provides an interface that systemd does not, users of said process manager should also be able to use it.)
Niche process manager support cannot come at significant burden for everyone else.
You are precisely defending the opposite. Systemd is the niche process manager here (even if the hugest one).
Generalized process manager support cannot come at significant burden for everyone else that wants to use their niche process managers.
Also, how portable is that systemd interface?
What do you mean by "portable"? Systemd only supports Linux currently.
Besides not being useful outside Linux, I was talking about the systemd as an abstraction.
In this sense I believe it is a good compromise. A general opt-in interface plus the possibility of writing raw files for direct use. AFAICT Nix provides this already.
Also you completely ignored some of systemd's features like service hardening, sandboxing, tmpfiles.d, journald, networkd. Reimplementing that all in shell script is probably possible but at what cost?
Why to reimplement anything? Where are these really used that are really important? Point of other service manager is to reduce what's not important.
Also, I believe having an alternating implementation added allows for future flexibility as well, maybe about another thing, before it deprecates, who knows?
Why to reimplement anything? Where are these really used that are really important? Point of other service manager is to reduce what's not important.
Because those features are extensible used in NixOS? NixOS is for example one of the frontrunners for systemd's LoadCredential, EnvironmentFile and family. Also services are using hardening in various ways like using BindPaths to make service variations easier https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/acme/default.nix#L359-L363 . It is also used to give various CAPs to services without giving them full root access.
Also configuring those things with plain flat config files is a perfect match for NixOS. Also having it for networking a lot better than doing it with scripts. It is way easier to transition between two states cleanly and without complicated conditions on the configures side.
And journald makes logging easier. You no longer need to convince every application where and how to write it's log files and when to rotate them. It automatically rotates files when the disk gets full and compresses them. Of course you can use logrotate but if the file is filled within a short timeframe, your system might just crash because you ran out of disk space.
Also, I believe having an alternating implementation added allows for future flexibility as well, maybe about another thing, before it deprecates, who knows?
but do we have actual viable alternative implementation that support the features we are using right now? If the alternatives only support the bare minimum of an init system and we need to take care of all the additional features used somehow ourselves then I am not sure if it actually is one.
That wouldn't be such a big deal if we wouldn't use those features but we are using them because they are convenient, save us time and allow us to focus on the actual service instead of worrying about shellscripts that start it.
Besides not being useful outside Linux, I was talking about the systemd as an abstraction.
Being a bit more precise:
- what does Systemd provide that no other similar software stack (like, say, the native NetBSD or FreeBSD or even GNU Shepherd from Guix) can do?
- how much can we live without those extras?
- for the particular cases the extras are needed, what does it affect the general case?
I appreciate and like the idea but adding extra layers of abstractions even for NixOS is not a good looking sign.
personally i agree with that statement and i think we shouldn't add a layer of abstraction on top of service managers - but i definitely think we should add a "portable service layer"... it just depends how you define that
as @infinisil and i discussed above:
I can imagine a
service.nixwhich declares a common set of options for a module and then several other files (systemd.nix,supervisord.nix,launchd.nix, etc...) which define different implementations of these options (maybe even declaring some implementation specific options of their own). I love how this could tie into RFC 140!
i think this is the only way to add more without creating a burden on people who only care about systemd implementations
Also you completely ignored some of systemd's features like service hardening, sandboxing, tmpfiles.d, journald, networkd.
It's worth mentioning that some systemd-* commands work well as standalone binaries, while some need a small patchset. Artix and Gentoo have been doing this for a long time.
Ah yes, that reminds me of, eudev and elogind, that has been forked out from systemd as standalone package by gentoo and its even used by void and other distributions.
About this extirpation from systemd:
https://guix.gnu.org/en/blog/2016/gnome-in-guixsd/
all alternatives to systemd lack major features of it is really worrying me.
I don't want to use systemd but it really does have the best features of all the init systems available. Something better than systemd is going to come along so we should be ready to pivot rather than have a messy transition with a lot of bad feelings, which happened with systemd and some of the other distros.
Doing the abstraction now avoids fighting and forking later.
Something better than systemd is going to come along
Some time ago we talked about initware, a portable implementation of systemd.
https://discourse.nixos.org/t/should-we-support-initware-is-a-fork-of-systemd-ported-to-different-bsd-systems/14400