Add `flake-parts` module

[srid]

Is there any interest in a flake-parts module that simplifies the following?

https://github.com/nammayatri/common/blob/main/nix/arion.nix

The flake-parts module would allow setting dockerCompose (see the above Nix file), and automatically produce the corresponding docker-compose-foo package that can be nix run'ed (in place of docker-compose or arion).

{ 
  perSystem = { pkgs, ... }: {
    arion.configs.foo = {
      project.name = "nammayatri-svc";
      services = {
        postgres = ... 
      };
    };
  };
}

Then nix run .#docker-compose-foo ... or nix run .#arion-foo.

Happy to open a PR for this.

[srid]

Comment from @roberth on Matrix, for reference:

[roberth]

The run approach may be interesting. It provides a bit of evaluation caching, which will be very welcome. I'm just not sure if it's the best kind of caching. Ideally, arion would store the generated compose file in a place where it's trivially in sync with the container engine's state; some existing metadata facility or some kind of volume or even container to hold on to that state. In that case almost all cache misses on nix run would just get in the way.

Anyway, here's the option

  --prebuilt-file JSONFILE Do not evaluate and use the prebuilt JSONFILE
                           instead. Causes other evaluation-related options to
                           be ignored.

[srid]

A naive and quick-n-dirty implementation: https://github.com/nammayatri/common/commit/77fbb94fd5ec5f7a31293e86e18335fef19e6990 & example

(Just to get something immediately working for sharing between our repos)

A better implementation would include possibly:

• A flake (rather than perSystem) option for dockerComposeConfiguration, since this is just a module.
• Support multiple docker-compose configurations, via flake.dockerComposeConfigurations (like flake.nixosConfigurations)
  • Correspondingly, produce multiple arion packages (that wrap --prebuilt-file)?

cc @aravindgopall

[roberth]

I would avoid calling it dockerCompose* because arion does a few things on top of docker compose. To be fair, just a few, but some nonetheless:

• Extra configuration, such as integration with NixOS
• Building and loading dockerTools images (potentially also nix2container at some point, if that's not integrated into dockerTools instead)
• A default exec command

Those are just the main ones off the top of my head. These are extensions of the docker compose format that are required by arion.

Furthermore, arion has multiple levels of modules that it supports

• arionProjectModules
• arionServiceModules
• nixosModules

• A flake (rather than perSystem) option for dockerComposeConfiguration, since this is just a module.

These are the definitions I use

• configuration: a completely "instantiated" configuration; not a single parameter remains
• module: anything from a single setting to a complete configuration, but not finalized like a configuration

Let's apply this to arion:

• arionProjectConfigurations.<name> in perSystem: a project that's ready to run on any given platform, somewhat like a docker compose file for each platform. This can not be extended.
• arionProjectConfigurations.<name> in top level (flake): a completely specific project that evaluates to a single compose file. This is a good representation of a production deployment or production-like deployment, but not suitable as a developer tool because they'll be stuck on one cpu+os platform
• arionProjectModule.<name> in perSystem: some settings or logic that were defined with definitions for some platform in scope. Both perSystem and arion modules take a system parameter, so this is taking two system parameters that should really be the same. It adds the inconvenience of having to specify the system at every import. Imports are not part the module fixpoint computation like all the other options, so this extra inflexible as well, if you do need to work with multiple systems.
• arionProjectModule.<name> in top level (flake): just a module; easily accessible and composable without context beyond the module system, nice and simple. It can make use of its implied system parameter (pkgs.stdenv.hostPlatform.system) to access perSystem of the defining flake using withSystem/getSystem or the derived, somewhat magical moduleWithSystem. (I've noticed that people wish for a simpler solution to this problem and they assume that is feasible, but when they come up with suggestions, they give up on the composability of modules. That's not something we're entitled to give up.)

• Support multiple docker-compose configurations, via flake.dockerComposeConfigurations (like flake.nixosConfigurations)

+1, except for dockerCompose* as discussed.

• Correspondingly, produce multiple arion packages (that wrap --prebuilt-file)?

Bundling the tool and the contents of the deployment makes heterogeneous multi-system remote deployment a little bit harder, if it's supposed to be an arbitrary host. However, if we take advantage of the perSystem and non-perSystem structure, we can still model this correctly for projects in arionProjectConfigurations: the system of the arion tool comes from the perSystem.apps context, whereas the system of the arion project is defined in the project modules.

[shivaraj-bh]

arionProjectModule. in perSystem: some settings or logic that were defined with definitions for some platform in scope. Both perSystem and arion modules take a system parameter, so this is taking two system parameters that should really be the same. It adds the inconvenience of having to specify the system at every import. Imports are not part the module fixpoint computation like all the other options, so this extra inflexible as well, if you do need to work with multiple systems.

@roberth does settings here refer to the settings similar to what you have in nixos-module.nix?

[roberth]

@shivaraj-bh more in an abstract sense. Change that to arionProjectModules.<name>, plural, and its type would be deferredModule, similar to nixosModules.

Actually we might even change that to modules.arionProject.<name>, to be in line with https://github.com/NixOS/nix/issues/6257.