NixOS
Functional tests fail when executing home-manager switch
I had home-manager setup in my Mac M4 and everything was working fine until I decided to add an additional program into my home manager flake. After I run switch, some nix-functional-tests fail:
3/196 nix-functional-tests:main / remote-store FAIL 1.52s exit status 1
37/196 nix-functional-tests:main / secure-drv-outputs FAIL 1.01s exit status 1
111/196 nix-functional-tests:main / impure-env FAIL 1.39s exit status 1
Any help is appreciated.
I have the same problem but on an m1, but when I run the same flake.nix on an m4 it works, although I don't know if it works because the base pkgs are already cached.
error: builder for '/nix/store/m9b6nc68f6bhz8dna9x92bwmk75nd70z-nix-2.28.2.drv' failed with exit code 3;
last 25 log lines:
> 187/196 nix-functional-tests:local-overlay-store / delete-refs SKIP 0.53s exit status 77
> 188/196 nix-functional-tests:local-overlay-store / delete-duplicate SKIP 0.55s exit status 77
> 189/196 nix-functional-tests:local-overlay-store / gc SKIP 0.60s exit status 77
> 190/196 nix-functional-tests:local-overlay-store / verify SKIP 0.62s exit status 77
> 191/196 nix-functional-tests:local-overlay-store / optimise SKIP 0.46s exit status 77
> 192/196 nix-functional-tests:local-overlay-store / stale-file-handle SKIP 0.41s exit status 77
> 193/196 nix-functional-tests:git-hashing / simple OK 4.10s
> 194/196 nix-functional-tests:flakes / non-flake-inputs OK 6.86s
> 195/196 nix-functional-tests:git-hashing / fixed OK 3.79s
> 196/196 nix-functional-tests:flakes / flakes OK 26.48s
>
> Summary of Failures:
>
> 4/196 nix-functional-tests:main / remote-store FAIL 1.08s exit status 1
> 40/196 nix-functional-tests:main / secure-drv-outputs FAIL 0.91s exit status 1
> 112/196 nix-functional-tests:main / impure-env FAIL 1.13s exit status 1
>
> Ok: 158
> Expected Fail: 0
> Fail: 3
> Unexpected Pass: 0
> Skipped: 35
> Timeout: 0
>
> Full log written to /private/tmp/nix-build-nix-2.28.2.drv-2/source/build/meson-logs/testlog.txt
For full logs, run:
nix-store -l /nix/store/m9b6nc68f6bhz8dna9x92bwmk75nd70z-nix-2.28.2.drv
error: 1 dependencies of derivation '/nix/store/9949pzryxj73jq82ymrmszgvrj3hfy4q-nixos-option.drv' failed to build
error: 1 dependencies of derivation '/nix/store/sql4wsbkbqz616inglhfpsp7ivgxs0mf-home-manager.drv' failed to build
I am seeing the same, have tried (in some desparation) both a Lix installer and the Determinate Systems package, on my M3.
Can one of you show us what the actual test failure is? From the full logs?
Please do not post more "me too" comments. Unless you can provide more error context i.e. full build logs or do you have a Nix sandbox enabled during the build.
I'm facing same issue but on m4 mac. Here is the full build logs attached.
Previously I had used the curl ... | sh determinate systems nix installer. I tried upgrading, this time by installing their new .pkg MacOS package. I'm not sure why, but after doing this my error went away.
I'm facing same issue but on m4 mac. Here is the full build logs attached.
I can replicate these tests failures on my M4 Macbook Pro.
Previously I had used the curl ... | sh determinate systems nix installer. I tried upgrading, this time by installing their new .pkg MacOS package. I'm not sure why, but after doing this my error went away.
Did the same, and I went from the original 14 test failures to 1-4 test failures (the tests seem flaky). See one sample output:
Summary of Failures:
177/201 nix-functional-tests:flakes / shebang FAIL 1.68s exit status 1
Ok: 168
Fail: 1
Skipped: 32
And then another run:
┃ > Summary of Failures:
┃ >
┃ > 50/201 nix-functional-tests:main / nix-shell FAIL 2.97s exit status 1
┃ > 130/201 nix-functional-tests:ca / nix-shell FAIL 1.44s exit status 1
┃ > 187/201 nix-functional-tests:flakes / shebang FAIL 2.23s exit status 1
Same issue. (M1 Pro)
I recall doing a garbage collect a few days ago so I am guessing some previously present packages are now gone causing this issue.
This is a show stopper for me currently. Any way to work around this without reinstalling (as the other comments seem to suggest)?
After more investigation, I was able to identify the root cause and create a fix for the test flakiness. The affected tests on my machine are:
┃ > 50/201 nix-functional-tests:main / nix-shell FAIL 2.97s exit status 1
┃ > 130/201 nix-functional-tests:ca / nix-shell FAIL 1.44s exit status 1
┃ > 187/201 nix-functional-tests:flakes / shebang FAIL 2.23s exit status 1
The test scripts (foo, bar, ruby) in tests/functional/shell.nix are created without shebangs:
https://github.com/NixOS/nix/blob/ccba158780b3d8957b7ef436a6cbc1ec08a0174e/tests/functional/shell.nix#L107-L112
This creates an executable file containing just echo foo with no interpreter line. On macOS, when these shebang-less scripts are executed via subshell capture like $(foo), they seem to intermittently produce empty output (I couldn't replicate this outside of the testing environment). It's possible this is macOS-specific behaviour, which would explain why it wasn't caught in CI.
To fix this, I added #!${shell} shebangs to each test script:
diff --git a/tests/functional/shell.nix b/tests/functional/shell.nix
--- a/tests/functional/shell.nix
+++ b/tests/functional/shell.nix
@@ -94,31 +94,34 @@ let
);
foo = runCommand "foo" { } ''
mkdir -p $out/bin
- echo 'echo ${fooContents}' > $out/bin/foo
+ echo '#!${shell}' > $out/bin/foo
+ echo 'echo ${fooContents}' >> $out/bin/foo
chmod a+rx $out/bin/foo
ln -s ${shell} $out/bin/bash
'';
bar = runCommand "bar" { } ''
mkdir -p $out/bin
- echo 'echo bar' > $out/bin/bar
+ echo '#!${shell}' > $out/bin/bar
+ echo 'echo bar' >> $out/bin/bar
chmod a+rx $out/bin/bar
'';
bash = shell;
bashInteractive = runCommand "bash" { } ''
mkdir -p $out/bin
ln -s ${shell} $out/bin/bash
'';
# ruby "interpreter" that outputs "$@"
ruby = runCommand "ruby" { } ''
mkdir -p $out/bin
- echo 'printf %s "$*"' > $out/bin/ruby
+ echo '#!${shell}' > $out/bin/ruby
+ echo 'printf %s "$*"' >> $out/bin/ruby
chmod a+rx $out/bin/ruby
'';
inherit (cfg) shell;
I did a 10 test builds with and without the patch, and it seems like without the patch, you see at least one test failure in around 50% of builds, and with the patch the issue seems to be entirely fixed. I tested this on macOS Sequoia 15.1 (M4), Nix 2.31.2, nixpkgs unstable (December 2024).
@lhohan @Aniruddha-Upadhya-K could you try applying this overlay to see if it fixes your problems?
nixpkgs.overlays = [
(final: prev: {
nix = prev.nix.appendPatches [
(final.writeText "fix-shell-nix-shebangs.patch" ''
diff --git a/tests/functional/shell.nix b/tests/functional/shell.nix
--- a/tests/functional/shell.nix
+++ b/tests/functional/shell.nix
@@ -94,21 +94,24 @@ let
);
foo = runCommand "foo" { } '''
mkdir -p $out/bin
- echo 'echo ''${fooContents}' > $out/bin/foo
+ echo '#!''${shell}' > $out/bin/foo
+ echo 'echo ''${fooContents}' >> $out/bin/foo
chmod a+rx $out/bin/foo
ln -s ''${shell} $out/bin/bash
''';
bar = runCommand "bar" { } '''
mkdir -p $out/bin
- echo 'echo bar' > $out/bin/bar
+ echo '#!''${shell}' > $out/bin/bar
+ echo 'echo bar' >> $out/bin/bar
chmod a+rx $out/bin/bar
''';
bash = shell;
bashInteractive = runCommand "bash" { } '''
mkdir -p $out/bin
ln -s ''${shell} $out/bin/bash
''';
# ruby "interpreter" that outputs "$@"
ruby = runCommand "ruby" { } '''
mkdir -p $out/bin
- echo 'printf %s "$*"' > $out/bin/ruby
+ echo '#!''${shell}' > $out/bin/ruby
+ echo 'printf %s "$*"' >> $out/bin/ruby
chmod a+rx $out/bin/ruby
''';
inherit (cfg) shell;
'')
];
})
];
As a note, incidentally, I found another bug (rm: command not found) in many of the tests that seems to be caused by a faulty trap definition on shell exit, but I decided not to fix it because it doesn't seem to be failing the tests.
I did a bit more digging trying to reproduce what's happening with the shebangs and created this reproduction script. Turns out that when the scripts are run without a shebang, they fail 15% of the time with a SIGSEGV violation. I can't speculate as to what's causing them, but adding the shebang seems like a safe cross-platform fix. I'll create a PR for it.