flake-utils icon indicating copy to clipboard operation
flake-utils copied to clipboard

featreq: function to filter out overlay packages for supported platforms

Open colemickens opened this issue 5 years ago • 12 comments

This is what I'm doing now. I know there are more sophisticated versions around that will check dependencies recursively. This seems eligible for flake-utils, any overlay or package set providers will likely want to use some sort of function like this.

(Bonus points if we discuss whether to filter unsupported platforms out entirely, or if we give a more helpful error in those cases.)

colemickens avatar Aug 01 '20 07:08 colemickens

{
      packages = let
        pkgs = sys: 
          let
            nixpkgs_ = (pkgsFor inputs.nixpkgs sys true);
            packagePlatforms = pkg: pkg.meta.hydraPlatforms or pkg.meta.platforms or [ "x86_64-linux" ];
            pred = n: v: builtins.elem sys (packagePlatforms v);
          in
            nixpkgs_.lib.filterAttrs pred nixpkgs_.waylandPkgs; 
      in {
          x86_64-linux = pkgs "x86_64-linux";
          aarch64-linux = pkgs "aarch64-linux";
        };
}

colemickens avatar Aug 01 '20 08:08 colemickens

With today's flake-utils you would achieve something similar like this:

{
  description = "colemicken's wayland overlay";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    {
      overlay = import ./overlay.nix;
    }
    //
    (
      flake-utils.lib.eachSystem [ "aarch64-linux" "x86_64-linux" ] (system:
        let
          pkgs = import nixpkgs {
            inherit system;
            config = { };
            overlays = [ self.overlay ];
          };
        in
        {
          packages = pkgs.waylandPkgs;
        }
      )
    );
}

Assuming that the overlay puts all its packages under pkgs.waylandPkgs, and that pkgs.waylandPkgs only contains derivations (otherwise nix flake check complains).

The big chunk at the end could probably be simplified for the common case.

{
  description = "colemicken's wayland overlay";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.simpleFlake {
      inherit self nixpkgs;
      overlay = import ./overlay.nix;
      overlayAttr = "waylandPkgs";
      systems = [ "aarch64-linux" "x86_64-linux" ];
    };
}

How does that look?

zimbatm avatar Aug 01 '20 10:08 zimbatm

https://github.com/numtide/flake-utils/pull/5

zimbatm avatar Aug 01 '20 11:08 zimbatm

Cool, I need to try it and glance at eachSystem but it looks like what I want, yeah. And I like the looks of #5. I'm going to try to re-visit my various flake repos next weekend, maybe dogfood #5 and then comment again / close this out.

colemickens avatar Aug 03 '20 00:08 colemickens

Looking forward to your feedback!

zimbatm avatar Aug 03 '20 08:08 zimbatm

I'm skimming through the source and #5, I'm not seeing anything that does the filtering I'm looking for based on the meta.platforms unless I'm looking in the wrong place?

colemickens avatar Aug 07 '20 02:08 colemickens

You're right. .flattenTree should be changed to take the system as an argument. Do you want to tackle this?

zimbatm avatar Aug 07 '20 09:08 zimbatm

I can't anytime soon. But cleaning up my flake.nixs will eventually block me and I plan to clean them via this repo, so I'll get to it eventually if you don't. Thanks for confirming my understanding and starting up this repo.

colemickens avatar Aug 07 '20 10:08 colemickens

We're slowly converting our overlays to flakes, and something that filters the per-system package set based on meta.hydraPlatforms would be very helpful.

In our current overlays, we import nixpkgs + "/pkgs/top-level/release-lib.nix" and use its functionality to build Hydra jobsets. Nix's built-in flake support, combined with flake-utils, replaces most of that machinery, except for the bit that filters out packages that don't support a particular platform.

dhess avatar Dec 22 '20 15:12 dhess

^ please try out this new implementation in PR 13

zimbatm avatar Dec 22 '20 19:12 zimbatm

@nrdxp do stars align here? (through PR13). I think nrdxp/nixflk is a complement use case to look for. Its flake is already quite complex while trying to solve common use cases around nixos configurations.

My reasoning is: flake-utils is an even higher order abstraction than nixflk, so moving things up the ladder (where adequate) might perfectly serve the presumptive goal of both projects to set a standardization precedents early on.

blaggacao avatar Dec 30 '20 15:12 blaggacao

My solution does check the buildInputs of each flake package for platform dependence, but not recursively.

nrdxp avatar Dec 31 '20 22:12 nrdxp