What language is flake.nix written in?

[dasJ]

This issue may sound silly but it's actually not clear to me what language they are written in. The syntax and file ending both look like Nix but the manual doesn't apply most of the time. The manual makes it seem like this should work:

{
  description = "Testflake";
  outputs = import ./outputs.nix;
}

but running that results in

error: expected a function but got a thunk at /nix/store/lm4pyqnkbiwvp2kxjd64nqb2xkdxmbfx-source/flake.nix:4:3

So what language are flake.nix files written in (is there any specification or documentation) and why is that language not Nix?

[edolstra]

It's a subset of Nix that doesn't allow computation in the flake metadata attributes. So e.g. outputs cannot be a function application like import ./outputs.nix, it must be a function directly outputs = { bla }: .... This is to prevent arbitrarily complex, possibly non-terminating computations while querying flake metadata.

This should be documented in the flake manpage.

[stale[bot]]

I marked this as stale due to inactivity. → More info

[dasJ]

Will care about this when the feature gets remotely usable for my use case so please leave this open

[exarkun]

I ran into this and was quite surprised and confused for about half an hour. I used let ... in ... at the top of my flake.nix and the resulting error message, "error: file '/nix/store/91d71n4sxkxnby3c3pbc8is4xm7a70r9-source/flake.nix' must be an attribute set" had me scratching my head (builtins.typeOf said it evaluated to an attribute set) and looking all over for someplace I had put some non-set type into a set position.

I didn't believe the error message was worded accurately and precisely because I am used to Nix error messages that tell me one type was expected in place of another with little or no further location information and I thought this was one of those cases.

An error message something like "file '...' must be written using the limited flake expression language" might have helped me discover my mistake more quickly.

Also, none of the docs I read on my way to learning about flakes said anything about this limited expression language so apart from tripping over this error I'm not sure how I would have learned about this.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/can-i-use-flakes-within-a-git-repo-without-committing-flake-nix/18196/33

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/experimental-does-not-mean-unstable-detsyss-perspective-on-nix-flakes/32703/2

[dzmitry-lahoda]

could be awesome if there would be turing decidable(always halt) security attack protected(like infinite reference circular import parsing) nix which allows to be flake input limited flake expression language and that to be documented as flake input.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flakes-and-pollution-of-downstream-projects/35411/8

[infinisil]

This issue has a better title and description, but it's a effectively a duplicate of https://github.com/NixOS/nix/issues/3966, which has a bunch more discussion.

[rhendric]

(@infinisil, did you mean #3966 maybe?)

[infinisil]

Ah yes thanks, fixed!

[matklad]

Let me attempt at a summary here.

The problem to solve here is providing access to flake metadata as a plain inert data, which only needs parsing, but not evaluation. If this is not done, than any operation with a flake requires evaluating nix code, which is slow and requires full nix tool.

A negative example for this problem would be the Python ecosystem, where running setup.py is (used to?) required to learn about package dependencies, which could make dependency resolution very slow (as the resolver must sequentially evaluate the graph of transitive dependencies)

A positive example would be Rust, where the metadata is available in inert .toml and additionally mirrored in the registry, and, as a result, the tooling is pretty snappy.

While inert metadata greatly speeds up and increases the reach of common use-cases, there are also uncommon use-cases which could make use of more computation power for metadata: example

Assuming we want to have inert metadata, there are two "dimension" for getting there:

• Split flake definition into two files, inert flake.jsontomlwhatever and flake.nix with code, OR use a single file, flake.nix
• Pick a language for inert subset, it could be JSON, TOML, or a data-only subset of nix

Status quo: a single file is used, with restricted subset of nix for metadata.

TOML support was attempted at here: https://github.com/NixOS/nix/issues/3966#issuecomment-706734581

[matklad]

Notes from myself:

Zig is designing a similar system at the moment. The current thinking is to have build.zig with code to describe how to build the current package, and build.zon (zig object notation) with data to express external dependencies. That is, two files, but using a data-only subset of the language

---

Disregarding practical implications and thinking from the first principles, using data-only subset of JavaScript (JSON) feels inferior to using data-only subset of nix.

[balazs-lengyel]

It seems to me that the opposing sides of this problem are not easily unify-able. on one hand dynamic behavior is really nice in some cases but on the other hand not everyone should always pay this cost.

One solution could be template programming:

• keep flake.lock and flake.nix as is
• add optional flake.template.nix which is written in fully featured nix
• add a nix flake template sub-command to pre-calculate arbitrary nix expressions from flake.template.nix and (over-)write the result to flake.nix.

this way we could:

• have a standard format (flake.nix) for tooling
• arbitrary nix expressions and time complexity in template
• fast info for flake users
• an easy way to script automatic template->flake updates, if needed
• can make flake.nix even more strict if needed for performance and recommend moving flake.nix to flake.template.nix if nix becomes too strict about flake evaluation.

what we lose:

• one more file and sub-command

(got here from experiencing https://github.com/NixOS/nix/issues/3966 and https://github.com/NixOS/nix/issues/4384)

[Cloudef]

Tried to be clever and hit this with:

  inputs = let
    dep = url: { inherit url; inputs.nixpkgs.follows = "nixpkgs"; };
  in {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    nix-darwin = (dep "github:LnL7/nix-darwin");
    home-manager = (dep "github:nix-community/home-manager");
    zig = (dep "github:Cloudef/nix-zig-stdenv");
    zls = (dep "github:zigtools/zls");
    hyprland = (dep "github:hyprwm/Hyprland");
    eww = (dep "github:elkowar/eww");
  };

[jorsn]

One solution could be template programming:

• keep flake.lock and flake.nix as is

• add optional flake.template.nix which is written in fully featured nix

• add a nix flake template sub-command to pre-calculate arbitrary nix expressions from flake.template.nix and (over-)write the result to flake.nix.

-- from https://github.com/NixOS/nix/issues/4945#issuecomment-1864538239

This is already possible by running the more complicated nix eval --raw -f flake.template.nix flake > flake.nix instead of nix flake template, where flake.template.nix contains something similar to the code below. For projects where the inputs are so numerous or complex that actual code is really helpful, it might be worthwhile to take this approach.

# flake.template.nix
with builtins;
let 
  toPretty = import (fetchurl
    "https://gist.githubusercontent.com/jorsn/012be3e868736359024007fc87b631cf/raw/40bd0d0183215cb6b2cf5ac801559faabee78a0a/prettyUnsafe.nix"
  );
  #or inherit (toPretty) (getFlake inputs.nixlib.url).lib.generators;
  #for less pretty output

  genFlake = thisFile: attrs:
    attrs // {
      flake =
        replaceStrings [ "\"<outputs>\"" ] [ "inputs: (import ${thisFile}).outputs inputs" ]
          (toPretty "" (attrs // { outputs = "<outputs>"; }))
        ;
    };
in genFlake ./flake.template.nix
{
  description = "description";

  inputs = let
    dep = url: { inherit url; inputs.nixpkgs.follows = "nixpkgs"; };
  in {
    nixlib.url = "github:nix-community/nixpkgs.lib";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    nix-darwin = (dep "github:LnL7/nix-darwin");
    home-manager = (dep "github:nix-community/home-manager");
    zig = (dep "github:Cloudef/nix-zig-stdenv");
    zls = (dep "github:zigtools/zls");
    hyprland = (dep "github:hyprwm/Hyprland");
    eww = (dep "github:elkowar/eww");
  };

  outputs = inputs: {
  };
}

For convenience, here is a link to the gist: https://gist.github.com/jorsn/012be3e868736359024007fc87b631cf

[roberth]

You could avoid some templating by treating flake.nix as mostly data, and then you put outputs = inputs: import ./outputs.nix inputs; at the end.

None of that should be needed though. We should consider actual solutions within flakes itself.

Plain data inputs?

Regardless of where and how we represent it, the major use case for this seems to be inputs. Does that really need to be plain data?

• With the current style of locking, yes, but with #7730, no.
• As a way of determining all (possible) source inputs of a flake: not unless we also forbid fetchTree and fixed output derivations. As long as we have those, it's not feasible to figure out all actual inputs anyway, and I don't think we should forbid those functions. In other words, such static analysis is not complete and therefore useless.
• As a way of bootstrapping fetchers that help fetch custom inputs, yes, but we don't have such a feature, so no.
• As a way of making sure that input declarations' evaluation doesn't require other inputs to be fetched, not really - we could just error out, or be clever about it.

So my conclusion thus far is that we could allow inputs to be proper expressions - not just data. After #7730 as mentioned.

Alternatively, in the lock file

Another, broader solution is to write non-data expressions' outcomes to the lock file. That satisfies the requirement of no evaluation, but it might go out of sync, so it'd be up to you to commit the lock file if you care about others consuming your metadata. Tooling such as CI can help, without being as intrusive as custom templating. Externally consumable flakes should have CI anyway.

[jorsn]

I wrapped my proposal in a flake, to improve ease of use:

nix flake init -t github:jorsn/flakegen
nix run .#genflake flake.nix

Even if we might have a better solution in the future, this is usable now, if anyone is desperate to program flake inputs.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/reference-diagram-for-nix-file-module-flake-function-use-cases-and-identification/43442/2

[sellout]

Rather than saying that a flake is “a subset of nix”, I think it’s generally helpful to describe a .nix file as containing either “a Nix expression” or “Nix data”[^1]. In practice it’s the distinction between whether the contents are evaluated before being acted on. A flake is not evaluated, but the handler for a flake will evaluate portions of it at different times.

[^1]: Fundamentally, a Nix expression is just Nix data where the data is a thunk of some type (() -> a), so it needs to be evaluated immediately (whereas, say, a module is Nix data of (roughly) type {…} -> {config = …, options = …}).

I think the richness of “Nix data” can be improved without compromising much, though. E.g.,

let
  name = "my-project";
  system = "x86_64-linux";
  nixpkgsRelease = "24.05";
in {
  description = name;

  outputs = inputs: {
    packages.${system}.${name} = …;
  };

  inputs.home-manager.url = "github:nix-community/home-manager/release-${nixpkgsRelease}";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-${nixpkgsRelease}";
}

should only require “normalization”, not evaluation.

Since outputs is a function, users should already be encouraged to use tools (like nix flake show and nix repl with :lf) to explore a flake, rather than opening the file directly, so adding a bit more abstraction shouldn’t introduce a problem there.

Now, defining (and implementing) what is part of normalization is another question (e.g., should normalization execute import).

But I think regardless of normalization, the data/expression distinction is important and serves a general purpose (and should be documented somewhere so that questions like this can be directed there). Then, instead of talking about “what language” a flake (or some other .nix file) is in, we talk about the “type” of the data.

[ranomier]

Just force the output to be a different file and let flake.nix be mainly just the input file.

I have this at the bottom of flake.nix. So much nicer.

  outputs = inArgs: import ./outputs.nix inArgs;

Then flake.nix is not a mixture of syntax.

[sellout]

@ranomier IMO, this issue isn’t about outputs in particular. For example, I just ran into another instance of this issue. I needed use my Cachix info somewhere else in my configuration, but I had to duplicate it because I’m not allowed to do

let
  cachix = {
    url = "https://….cachix.org";
    publicKey = "…";
  };
in {
  nixConfig = {
    extra-substituters = [cachix.url];
    extra-trusted-public-keys = [cachix.publicKey];
  };
  outputs = inputs: import ./outputs.nix {inherit cachix inputs;};
}

As I mentioned in my previous comment, I feel like any Nix file should at least be normalized[^2] first. Treating it like JSON[^3] discards most of the benefits.

[^2]: What “normalized” means needs to be specified (I don’t think there is anything between parse and eval in the current code, but I’m just a spectator). let is the main thing. If I can just extract a few constants like above, I’m pretty good. nixpkgs.lib wouldn’t be available, but should there be some partial evaluation of builtins? I would say it shouldn’t allow rec. If it allows import, that probably needs to be restricted (are circular imports already disallowed?). Does string interpolation happen? But maybe even restricted, it’s hard to eliminate all recursion loopholes – so then is it just a JSON structure, or do we just hope there are enough boundaries that people don’t stumble into them? (I would vote for the latter.)

[^3]: Even YAML has its weird anchors.

[ranomier]

@sellout Thank you for your time. I think i should clarify what my point is. It doesn't contradict with what we allow within the static flake part.

What i am trying to say is, whatever it is that is allowed within the flake part of that file, let's keep it within a file. Then we can say a flake.nix (maybe call it sources.nix or input.nix instead) is this particular format.

And let's keep the "output" also in a separate file.

From a UX and learning experience, it would be soooooo much easier to say things like.

• flake.nix is more static
• flake.nix doesn't run code
• In the output.nix you can run code
• etc ...

Additionally huge hurdle for me was and is to learn the custom language and being able to use more simpler language would be a really helpful thing.

---

2. yaml is not a good example^^ https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell

[stepbrobd]

weird question

if i submit a patch that allows arbitrarily complex flake.nix file under a flag say --trust-me-bro-it-will-terminate, what are the chances of it getting accepted? i already have a scratch PoC ready but definitely not ready for upstreaming

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-set-top-level-global-variable-in-flake-nix/72157/3

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/dynamically-calculate-the-value-of-a-string-in-nixpkgs/72378/3

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/outlining-the-differences-between-flakes-and-nix-configs/72996/1

[roberth]

I've opened https://github.com/NixOS/rfcs/pull/193 to explore a solution to this.