nix
nix copied to clipboard
NixOS
flakes: nix shell (and similar) fails, if flakes.nix inside a git-repo ist no yet added to the repo (error: getting status of '/nix/store/.../flake.nix': No such file or directory)
If a flakes.nix file is copied to a repository (empty or with content), then without adding it yet to the reposiroty, envoking nix shell and other flake related commands will fail with an error like
error: getting status of '/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source/flake.nix': No such file or directory
rather than falling back to just use the locally available flake.nix (should be the default IMH).
Steps To Reproduce
git init test && cd test- Copy any flake.nix file in there (see e.g. my attached one)
- run
nix shell - See error
Expected behavior
A nix shell should be opened using the locally present but not yet added or commited flakes.nix.
nix-env --version output
nix-env (Nix) 2.8.1
Additional context
The flake.nix file, that I used
{
inputs.artiq.url = "git+https://github.com/m-labs/artiq.git";
inputs.extrapkg.url = "git+https://git.m-labs.hk/M-Labs/artiq-extrapkg.git";
inputs.extrapkg.inputs.artiq.follows = "artiq";
outputs = { self, artiq, extrapkg }:
let
pkgs = artiq.inputs.nixpkgs.legacyPackages.x86_64-linux;
aqmain = artiq.packages.x86_64-linux;
aqextra = extrapkg.packages.x86_64-linux;
in {
defaultPackage.x86_64-linux = pkgs.buildEnv {
name = "artiq-env";
paths = [
# ========================================
# EDIT BELOW
# ========================================
(pkgs.python3.withPackages(ps: [
# List desired Python packages here.
aqmain.artiq
#ps.paramiko # needed if and only if flashing boards remotely (artiq_flash -H)
#aqextra.flake8-artiq
# The NixOS package collection contains many other packages that you may find
# interesting. Here are some examples:
ps.pandas
ps.numpy
ps.scipy
#ps.numba
#ps.matplotlib
# or if you need Qt (will recompile):
#(ps.matplotlib.override { enableQt = true; })
ps.matplotlib.override { enableGtk3 = true; }
ps.pybase64
#ps.bokeh
#ps.cirq
#ps.qiskit
]))
#aqextra.korad_ka3005p
#aqextra.novatech409b
# List desired non-Python packages here
#aqmain.openocd-bscanspi # needed if and only if flashing boards
# Other potentially interesting packages from the NixOS package collection:
#pkgs.gtkwave
#pkgs.spyder
#pkgs.R
#pkgs.julia
# ========================================
# EDIT ABOVE
# ========================================
];
};
};
}
Summary: I'm not aware of a simple and easy solution to this UX annoyance that does not lead to more UX problems elsewhere.
This is intended. While making this new behavior default would simplify a few situations, it will also make others more complicated. As an analogy to another technology that purports to provide reproducible source code trees (git) that does not provide a similar default behavior:
- git init test && cd test
- copy any file in there
- run
git showSee error
Perhaps the error message should be better, or there should be an easier way than git --intent-to-add to do what you'd like. Perhaps it should automatically include it, but with a warning that clarifies what is happening, but then we must be careful to avoid this becoming depended on and then authors accidentally leave files out of repos that are needed for evaluation to occur. Then all this behavior needs to be explained. For now, the rule is simple: "if you are in a git repo, flake evaluation only utilizes files that git is tracking."
Honestly the fact that flakes don't automatically add files is a feature, and for me it's one of the biggest draws because of its interaction with the eval cache.
Sure sometimes I forget to add a file, but I almost always forget to gitignore things like result in a fresh repo. If files were auto added without a flag or option, the eval cache would trigger rebuilds as often as nix-build used to.
Summary: I'm not aware of a simple and easy solution to this UX annoyance that does not lead to more UX problems elsewhere.
This is intended. While making this new behavior default would simplify a few situations, it will also make others more complicated. As an analogy to another technology that purports to provide reproducible source code trees (git) that does not provide a similar default behavior:
- git init test && cd test
- copy any file in there
- run
git showSee error
I can understand, that this is by design and intended. I can even live with that decision. Regarding your next statement...
Perhaps the error message should be better, or there should be an easier way than
git --intent-to-addto do what you'd like.
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX. Like
error: getting status of '/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source/flake.nix': No such file or directory
Local flake.nix found, which is not member of the repository. Consider adding it first.
or something similar.
Honestly the fact that flakes don't automatically add files is a feature, and for me it's one of the biggest draws because of its interaction with the eval cache.
Sure sometimes I forget to add a file, but I almost always forget to
gitignorethings likeresultin a fresh repo. If files were auto added without a flag or option, the eval cache would trigger rebuilds as often asnix-buildused to.
Maybe (or certainly) I do not understand the full nix architecture in detail yet.
But I was rather referring to making flakes not fail when a flakes.nix file certainly is there (but not a member of a repo yet). Not automatically adding it to the repo.
Yet, as stated one post earlier, I can live with that behavior/design choice. The error message should just be more clearer, so that someone without a deep understanding of nix, flakes etc can troubleshoot the situation with a simple git add flakes.nix.
I have the very same issue. My use case is this https://nixos.wiki/wiki/Flakes#Super_fast_nix-shell
I am using this for repos that might also be used by non-nix users that would get annoyed by adding shell.nix and flake.* files to the repos.
In the old world, I had my shell.nix in .gitignore (no flake.nix) and everything worked fine.
Also adding the flake.nix file to .gitignore will not be enough to get rid of "flake.nix': No such file or directory".
I have the very same issue. My use case is this https://nixos.wiki/wiki/Flakes#Super_fast_nix-shell I am using this for repos that might also be used by non-nix users that would get annoyed by adding
shell.nixandflake.*files to the repos. In the old world, I had myshell.nixin.gitignore(noflake.nix) and everything worked fine. Also adding theflake.nixfile to.gitignorewill not be enough to get rid of "flake.nix': No such file or directory".
Ignoring flake.nix would prevent your flake from ever being available. Definitely don't add that to your ignores.
I think you misunderstood my comment about how legacy UI commands would cause spurious rebuilds because unimportant files were added to the working dir. The common workaround was various source filters, notable nix-gitignore. The old workflow was basically "clone a repo, extend .gitignore ( to the point that it's almost identical to git clean -xfd ), create *.nix using nix-gitignore filters, and THEN run nix-*.
Skipping these steps would cause Nix to rebuild projects needlessly, often to such a degree that it was easier for Nix files to be maintained externally. The new UI'S git integration resolves these issues. The tradeoff is "I don't have to do all that filtering, I just have to git add flake.{nix,lock} once and forget about it". 😉
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX.
There is a pending PR which has a better message. What do you think about this? https://github.com/NixOS/nix/blob/5f1340219b83f15a4354aad94467ae642a1196ed/src/libfetchers/git.cc#L756-L759
I am absolutely for that. I spent about two hours debugging this and I am and will not be the last one falling for that. The error message should be way more clearer for a better UX.
There is a pending PR which has a better message. What do you think about this?
https://github.com/NixOS/nix/blob/5f1340219b83f15a4354aad94467ae642a1196ed/src/libfetchers/git.cc#L756-L759
This is a welcome solution for me!
I've also lost several hours to this issue. Since it wasn't completely non-trivial to find, the PR that contains this change is https://github.com/NixOS/nix/pull/6530. That PR is also still a draft, so I'd prefer this to stay open until it's merged.
@tomberek / @edolstra could one of you reopen this issue and put a "Closes #6642" in the PR description for #6530?
Like @WhatisRT, I lost hours to this issue. @tomberek Please follow @WhatisRT's suggestion and reopen the issue. Thank you.
This situation might be improved by making the nix CLI default to path:. instead of git+file:. when flake.nix is not added to git and/or is gitignored.
The usecase of adding a local flake.nix to a repo which would not accept it being committed upstream would be streamlined this way, though it wouldn't stamp out the unintuitive issues related to files not existing due to being untracked or ignored by git. It might even add to them, since adding flake.nix would change the behavior of other files in this regard.
That error message is indeed terrible UX, it provides no clue as to what's going on.
This situation might be improved by making the nix CLI default to
path:.instead ofgit+file:.whenflake.nixis not added to git and/or is gitignored.The usecase of adding a local
flake.nixto a repo which would not accept it being committed upstream would be streamlined this way, though it wouldn't stamp out the unintuitive issues related to files not existing due to being untracked or ignored by git. It might even add to them, since addingflake.nixwould change the behavior of other files in this regard.
Now this I actually agree with.
For clarity though you should add flake{nix, lock} to your repo. Yes the error message could be better ( PR it ).
This situation might be improved by making the nix CLI default to
path:.instead ofgit+file:.whenflake.nixis not added to git and/or is gitignored.
in many situations that would also lead to copying lots of data into the nix store, which i would find to be a regression. people would have to readjust their habits to explicitly state git+file. which i certainly feel resistance towards (i.e. laziness :laughing:).
since in the case of a dirty git tree users already get a warning, i suggest to also print a warning if flake.{nix,lock} exist but are not racked in git.
in many situations that would also lead to copying lots of data into the nix store, which i would find to be a regression. people would have to readjust their habits to explicitly state git+file. which i certainly feel resistance towards (i.e. laziness ).
It would only cause that in situations where currently it fails completely. When flake.nix is known to git, it would still default to git+file:., just as it does now.
Moving from non-functionality to poor performance doesn't seem like a regression to me.
It would only cause that in situations where currently it fails completely. When
flake.nixis known to git, it would still default togit+file:., just as it does now.
ah, i interpreted your suggestion as changing the default unconditionally. but you meant it to be dependent on the flake.{nix,lock} tracking status, did i get it right now? if so it'd deem that acceptable, however still somewhat of a foot-gun. how about in that case nix just acts as if flake.{nix,lock} where tracked?
Yes, that's what I meant.
Indeed, it might be better to just pretend flake.{nix,lock} are tracked even when they aren't. I haven't thought it all through, but it does seem like that has less potential fallout, and avoids copying .git to the store all the time, too.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/doing-the-zero-to-nix-and-get-a-random-error/27692/8
2 years since the issue creation and we still cannot run nix develop when the flake.* files are in gitignore? This is my company's app and do not want to commit these flake files which are not related to the project and which are only used by me. Just let me do what I want instead of imposing me this decision...
The problem is that there isn't really a good solution here.
If nix copies non-git files generally, then you end up with all kinds of detritus and intermediate files copied into the nix store, and then those intermediate files often mess up build processes because the timestamp data is gone.
If you build only from the latest actual commit then it's too annoying to quickly test out a change to a flake.
The compromise we've landed on is to use the dirty copies of any files tracked by git, but ignore other files. It's not perfect, but it's quite hard to come up with a change to it that wouldn't cost too much in terms of other unintuitive results. I'd love to see a nice solution to the usecase of adding a local flake.nix to a repo that doesn't have one upstream, but any such solution needs be considered in terms of all the effects it would have, not just that it solves that one issue.
I don't think we should be forced to overload .gitignore to serve this separate purpose of defining what goes into the nix store. Maybe have that as a default, but it needs to be overridable. I just tried using nix flakes for a new project for the first time and this is the very first use case I wanted. I think using flake.nix to nixify a project without having to check the flake.nix file into the repo is a very important use case.
I hit the same issue and was very confused by this, Fixing the error message as per the above would have really helped me and would be a very welcome change. This is an issue that should be well documented in flakes documentation. But this is the first time I have seen this behavior documented.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/problem-running-nix-develop/52249/3
I hit the same issue and was very confused by this, Fixing the error message as per the above would have really helped me and would be a very welcome change. This is an issue that should be well documented in flakes documentation. But this is the first time I have seen this behavior documented.
this seems to have been fixed in https://github.com/NixOS/nix/pull/9398 but sadly isnt in the nix that ships with 24.05 :/
I hit the same issue and was very confused by this, Fixing the error message as per the above would have really helped me and would be a very welcome change. This is an issue that should be well documented in flakes documentation. But this is the first time I have seen this behavior documented.
this seems to have been fixed in #9398 but sadly isnt in the nix that ships with 24.05 :/
This looks like a regression caused by this PR: https://github.com/kolloch/nix/commit/598deb2b23bc59df61c92ea25745d675686f3991
Since then the whole flake system also was under heavy development and refactoring.
Folks, pardon my harshness, but it's absolutely insane that this issue exists in the first place.
Let's forget about the confusing error message, and assume that it's fixed at some point (though it still isn't on 25.05beta767267.6607cf789e54). The bigger question is: why is Nix depending on the state of my Git repository to begin with??
I'm sure the answer is related to some deep technical reason regarding reproducibility, but surely the tool is able to figure out which files it needs for the build based on the specified inputs?
In my case I have a dead simple flake.nix:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }:
let
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
in {
devShells.${system}.default = pkgs.mkShell {
buildInputs = [
pkgs.nodejs_23
];
};
};
}
This doesn't depend on any files in the repo. The only thing Nix needs to do is to fetch nodejs_23 and spawn a new shell. That's it.
And even if it did depend on a file in the repo, then couldn't just those specific files be added to the input, or build context, or however you call it?
I'm sure that Docker is laughed at in this community, but when I run docker build . (using BuildKit), the only files included in the build context are ones referenced in the Dockerfile. Even before BuildKit, users had the option to use .dockerignore to avoid copying large files. Why can't Nix use a similar approach?
Honestly, it's these types of issues that make the Nix ecosystem so user hostile. We can argue all day whether this works as intended or not, but it objectively leads to a poor user experience. I just wish that developers of tools like this, with undeniably great technical ideas and talent, prioritized UX over everything else. :face_exhaling:
