nix
nix copied to clipboard
NixOS
nix-build: requires non-existent output 'out' from input derivation
Describe the bug
Packages build fails very often:
checking for true... true
error (ignored): error: cannot unlink '/tmp/nix-build-python3-minimal-3.9.12.drv-1/Python-3.9.12': Directory not empty
error: derivation '/nix/store/6xjmvr03428342gs1432ac0mn8myxnpc-grub-config.xml.drv' requires non-existent output 'out' from input derivation '/nix/store/88vs9d74w69j14s611vb5v7cn0
...
ran test tests/config.sh... [PASS]
installCheckPhase completed in 40 seconds
error: derivation '/nix/store/cqmxz06ggrnhr7wzf6s1kf39hykmsh6g-local-cmds.drv' requires non-existent output 'out' from input derivation '/nix/store/dw0x90b23k67mkav4bf24r63nvn97dir-nix-2.8.1.drv'
...
error: derivation '/nix/store/shnrbk0l8qdsfmvf4x6qp6nlvc0kdfjc-libmodsecurity-3.0.6.drv' requires non-existent output 'out' from input derivation '/nix/store/nrpsq5ld8fjp3adpimdiwkmg7wbhqsq3-lmdb-0.9.29.drv'
...
checking for references to /build/ in /nix/store/90wv2xdvci79wnj8nxlrxy73j1hiksjh-bind-9.18.3-host...
error: derivation '/nix/store/irc43zk5n8w2bp7ig0d5hlfi4ybm3vs9-system-path.drv' requires non-existent output 'host' from input derivation '/nix/store/ll61r00n2f62qkxlll7y0ba45mzby2m4-bind-9.18.3.drv'
I have to restart the build process manually each time. The problem happens on nix versions 2.8.1 and 2.9.0pre20220512_d354fc3 On version 2.8.0 the problem rarely occurred
I have to restart the build process
same here with
nix (Nix) 2.9.0pre20220505_f4102de and
nix (Nix) 2.9.1
i have to call nix-build . -A qt6.qtwebengine 3 times to fix
error: derivation qtpositioning requires non-existent output 'out' from input derivation qtserialport
this is after nix garbage collect = empty store. no problem with populated store
build log
nix-build.qt6.qtwebengine.nix-bug-6572.log
input: asjckii7hs9mpm3f623vyhb11hk9qiv6-source == vg85l7pk2cphsb7wbvp4j29xsa449ji8-source.drv
build: bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv
relevant logs
$ nix-build . -A qt6.qtwebengine -vvvv
checking substituter 'https://cache.nixos.org' for path '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source' checking substituter 'https://cache.nixos.org' for path '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source'
substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': created substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': woken up
building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: waitee 'building of '/nix/store/426q49fmidvdqgfjd68mpr1sl1clvwyd-xkeyboard-config-2.33.drv!out' from .drv file' done; 3 left building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: waitee 'building of '/nix/store/syz2smdcik9b1asd1pbcddbz5pgzqqm9-xrandr-1.5.1.drv!out' from .drv file' done; 2 left building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: waitee 'building of '/nix/store/l25hhx6c3jdgkh2vi58k3mf2fbvs0377-zlib-1.2.12.drv!dev' from .drv file' done; 1 left
substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': init substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': trying next substituter substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': trying next substituter path '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source' is required, but there is no substituter that can build it substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': done
building of '/nix/store/vg85l7pk2cphsb7wbvp4j29xsa449ji8-source.drv!out' from .drv file: waitee 'substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source'' done; 0 left
substitution of '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source': goal destroyed locking path '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source' lock acquired on '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source.lock' removing invalid path '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source'
setting builder env variable 'out'='/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source' scanning for references for output 'out' in temp location '/nix/store/vg85l7pk2cphsb7wbvp4j29xsa449ji8-source.drv.chroot/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source'
lock released on '/nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source.lock'
building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: waitee 'building of '/nix/store/vg85l7pk2cphsb7wbvp4j29xsa449ji8-source.drv!out' from .drv file' done; 0 left
building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: all inputs realised building of '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv!out' from .drv file: trying to build
error: builder for '/nix/store/bj4f7h30ak3k65b3n3zidwyr6gcbsbdr-qtwebengine-6.3.1.drv' failed with exit code 1; > build input /nix/store/asjckii7hs9mpm3f623vyhb11hk9qiv6-source does not exist
the "removing invalid path" looks suspicious
My guess would be nix starting the next build too early before the outputs of the previous build is properly registered.
aka build input X does not exist #1744
in something simple as
{ pkgs }:
with pkgs;
let
x = fetchFromGitHub {
# ...
};
in
stdenv.mkDerivation {
buildInputs = [ x ];
# ...
}
this time, restarting the build does not help
i have to manually eval the first derivation to fix this
these 2 derivations will be built:
/nix/store/g60qiv2b65238a6f4x32abm4n25028a1-source.drv
/nix/store/dvblz3fi0x3227kj393y0vcmkann8rg6-qtwebengine-6.3.1.drv
building '/nix/store/g60qiv2b65238a6f4x32abm4n25028a1-source.drv'...
trying https://github.com/milahu/gnumake-tokenpool/archive/c85ddf72c69ccea41f840e5fcfa58ee6719aa038.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 103k 0 103k 0 0 140k 0 --:--:-- --:--:-- --:--:-- 140k
unpacking source archive /build/c85ddf72c69ccea41f840e5fcfa58ee6719aa038.tar.gz
building '/nix/store/dvblz3fi0x3227kj393y0vcmkann8rg6-qtwebengine-6.3.1.drv'...
build input /nix/store/l2vvz50anjhw7g1l9xkzjcs5k71nqw11-source does not exist
error: builder for '/nix/store/dvblz3fi0x3227kj393y0vcmkann8rg6-qtwebengine-6.3.1.drv' failed with exit code 1;
last 1 log lines:
> build input /nix/store/l2vvz50anjhw7g1l9xkzjcs5k71nqw11-source does not exist
For full logs, run 'nix log /nix/store/dvblz3fi0x3227kj393y0vcmkann8rg6-qtwebengine-6.3.1.drv'.
nix-build -E 'with import <nixpkgs> {}; with pkgs; fetchFromGitHub {
owner = "milahu";
repo = "gnumake-tokenpool";
rev = "5dabefe12144bb91b3bf9b8ec67004282e6f0f18";
sha256 = "6CfKYQgYSzR/8Nule7gWkwn0W9crqlWcgPIr39LUYsk=";
}'
this works as expected
nix-build -E 'with import <nixpkgs> {}; with pkgs; let x = stdenv.mkDerivation { name = "x"; unpackPhase = "sleep 999"; }; in stdenv.mkDerivation { name = "y"; buildInputs = [ x ]; }'
aka do not know how to unpack source archive
unpacking sources unpacking source archive /nix/store/2nwk707f9w8ydnqh10hwq54bxdm3cgl4-source do not know how to unpack source archive /nix/store/2nwk707f9w8ydnqh10hwq54bxdm3cgl4-source
$ ls /nix/store/2nwk707f9w8ydnqh10hwq54bxdm3cgl4-source ls: cannot access '/nix/store/2nwk707f9w8ydnqh10hwq54bxdm3cgl4-source': No such file or directory
... where source is a fetchFromGitHub
This issue is wreaking havoc on our CI. Fortunately it turns out that it is at least perfectly reproducible there, so I was able to bisect it using the Hydra binary tarballs down to the merge of https://github.com/NixOS/nix/pull/6227 /cc @edolstra. Which at least seems plausible as that PR directly changed the conditional leading to the error: https://github.com/NixOS/nix/pull/6227/files#diff-5784d1ad7c22601c75b86872534049bea5173377de5d61b7446452beabc16d3fL491-R534
Hopefully this is somewhat helpful, but I'm experiencing this (1) seemingly more lately, (2) almost every time I have a large cross-compile rebuild. Not sure if it's the cross-compile part, the large build part, or if it's a misnomer. I know that when I cross rebuild my risc-v config, I basically need to instead run until ./build-that.sh; do sleep 1; done to get it to actually keep progressing and eventually finish the build.
edit: my current nixbuild.sh wrapper looks like this: (it retries for this error, "signal 9", etc): https://github.com/colemickens/nixcfg/blob/bb78cb9e42d4a78d963548e52c948aaf781292e5/misc/nixbuild.sh#L48-L74
I am suffering from this too and downgraded to nix 2.7.
Same here but now I hit #6816 (which I can patch around) and #6253.
Edit: I guess
while true; do nixos-install; doneis a solution for now.
~~Not completely. I sometimes like to stress-test my computer and nixpkgs by running nixos-rebuild --no-net, and after looping for a while I was unable to build anything really because any build would fail with "too many open files". So I suspect when this bug is triggered and the build is aborted, Nix is not closing open files properly.~~
EDIT: Disregard, I can trigger this from a cold boot.
We hit this in CI where looping forever is not an option.
On Sun, Oct 2, 2022, 10:33 PM endgame @.***> wrote:
Edit: I guess while true; do nixos-install; done is a solution for now.
Not completely. I sometimes like to stress-test my computer and nixpkgs by running nixos-rebuild --no-net, and after looping for a while I was unable to build anything really because any build would fail with "too many open files". So I suspect when this bug is triggered and the build is aborted, Nix is not closing open files properly.
— Reply to this email directly, view it on GitHub https://github.com/NixOS/nix/issues/6572#issuecomment-1264853774, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBACRF7FXIDP5MLUQOU4ODWBJAX3ANCNFSM5W6EUWFA . You are receiving this because you commented.Message ID: @.***>
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/content-addressed-nix-call-for-testers/12881/206
My rough guess is that this is one of the many bugs that arose with the implementation of impure derivations https://github.com/NixOS/nix/pull/6227
@Ericson2314 : You added this portion back in 2020: https://github.com/NixOS/nix/commit/2de201254e8669a6768bb739ff27fd94a87a71b9
The 6227 PR removed that fix. But the equivalent fix from that commit near (https://github.com/NixOS/nix/pull/6227/files#diff-5784d1ad7c22601c75b86872534049bea5173377de5d61b7446452beabc16d3fR412) was not removed. So it seams either both are need or neither. But i don't see a logic error in the translation between the two.
@Kha you mentioned you had a reproducer?
I am also able to reproduce this on our CI system. I have a build that fails with this every time. It always seems to be the same derivation failing with this error as well. Will have a look at the details, to see which one.
I have a derivation that builds nearly every x86_64-linux derivation that I care about, and I patch mesa. So... basically anytime nixos-unstable moves, I hit this. Enough that I script around it. I can add some extra logging to save the logs from when this case hits and see if there's a pattern
/nix/store/mw4cfrgqgjyvgbvbkp4fjy1jh16yv3rk-stage-2-init.sh.drv
/nix/store/90y1xi74f5llb1rvr0zi9vvmjxj6nfmb-nixos-system-openstick-22.11.20221009.8eb771c.drv
copying 0 paths...
building '/nix/store/jwrg88l48lpk3i1l9d6kqkaz9fns2sil-lvm2-aarch64-unknown-linux-gnu-2.03.16.drv'...
building '/nix/store/aqf6nk4css0na23r11v17i50xwk4rx1v-libusb-aarch64-unknown-linux-gnu-1.0.26.drv'...
building '/nix/store/3faqh8w9rpljbal5qxmhf8bwgp83013j-dbus-aarch64-unknown-linux-gnu-1.14.0.drv'...
building '/nix/store/hzgpimk3rmhh6zwv6pgrqv669n44kc0p-qemu-7.1.0.drv'...
building '/nix/store/yli6hcfc878ny59691v8ckyc35hzaiwl-usbutils-aarch64-unknown-linux-gnu-014.drv'...
building '/nix/store/q4pmhgdc1f2hg8m7f6py49cm2zv20bik-dbus-glib-aarch64-unknown-linux-gnu-0.112.drv'...
building '/nix/store/q6pxbwdzhwpf0p4mv2hnrz1frs15vj6c-python3.10-dbus-python-1.2.18-aarch64-unknown-linux-gnu.drv'...
building '/nix/store/rlyiaqswagpnclqbf1r44n4isq76zmgv-cryptsetup-aarch64-unknown-linux-gnu-2.5.0.drv'...
error (ignored): error: cannot unlink '/tmp/nix-build-qemu-7.1.0.drv-3/qemu-7.1.0/build/docs': Directory not empty
error: derivation '/nix/store/dwg4qirqz6dvgax0fkwlnpilv3byi562-systemd-aarch64-unknown-linux-gnu-251.4.drv' requires non-existent output 'out' from input derivation '/nix/store/rlyiaqswagpnclqbf1r44n4isq76zmgv-cryptsetup-aarch64-unknown-linux-gnu-2.5.0.drv'
==:: nixbuild: build: retry (bug: nixos/nix#6572)
==:: nixbuild: build (on: auto) (log: /tmp/nixbuild-Zb2sdemU)
these 134 derivations will be built:
@colemickens my theory is that there's a race which is only triggered when a derivation D is scheduled to be built directly after one of its immediate inputs I. It might be necessary for I to be fairly large, so that when D begins its build, I hasn't finished being copied/registered/otherwise-set-up (not sure of the details) into its final position in the store. If the build scheduling allows for something else to happen in between, or if I is small, it gets set up in the store before D goes looking for it.
still no luck to reproduce ...
test.sh
#! /bin/sh
cat >/dev/null <<EOF
build 2 expressions: small + large
small is input for large
reproduce
nix-build: requires non-existent output 'out' from input derivation
https://github.com/NixOS/nix/issues/6572
dependencies
nix-shell -p unixtools.xxd git
workaround in /etc/nixos/flake.nix
nix.package = pkgs.nixVersions.nix_2_7;
bug is in nix 2.8 and later
EOF
# size of the "large" expression
#nix_body_size=10
nix_body_size=1000 # 5 seconds
#nix_body_size=5000 # 15 seconds
#nix_body_size=10000 # 50 seconds
# test version. force rebuild
version=$(date +%F.%H-%M-%S)
( set -x; nix --version )
use_git=true
git_port=8120 # random
use_http=false
http_port=8502 # random
http_host=127.0.0.1
if $use_git
then
echo preparing git repo
#git_dir=$(mktemp -d)
git_base_path=git-repos
git_path=test
git_dir=$git_base_path/$git_path
rm -rf $git_dir || true
mkdir -p $git_dir
git -C $git_dir init
echo "version=$version" >$git_dir/test.txt
git -C $git_dir add test.txt
git -C $git_dir commit -m init
#git -C $git_dir log # debug
git_host=127.0.0.1
# todo
cp -r $git_dir ${git_dir}.2
rm -rf ${git_dir}.2/.git
git_sha=sha256-$(nix-hash --type sha256 ${git_dir}.2 | xxd -r -p | base64)
rm -rf ${git_dir}.2
echo serving git repo git://$git_host:$git_port/$git_path
echo checking port $git_port
echo "netstat -nvep | grep -w $git_port"
netstat -nvep 2>/dev/null | grep -w $git_port
echo starting git server
git_pid_file=git-daemon.pid
set -x
if ! git daemon --listen=$git_host --port=$git_port --detach --pid-file=$git_pid_file --base-path=$git_base_path --export-all
# --verbose --informative-errors
then
echo "error: failed to start git server"
fi
set +x
sleep 1 # wait for server
echo started git server
fi
if $use_http
then
echo starting http server
if ! which sirv >/dev/null
then
echo "please install the sirv http server:"
echo "npm install -g sirv-cli"
exit 1
fi
[ -d public ] || mkdir public
# dev mode = no caching
# probably a caching server like "python3 -m http.server"
# would work too, as we kill the server after use
sirv public/ --no-clear --dev --host $http_host --port $http_port &
http_pid=$!
sleep 1 # wait for server
echo started http server with pid $http_pid
(
cd public/
test_path=test.txt
dd if=/dev/urandom of=$test_path bs=1024 count=4 status=none
test_sha=$(sha256sum $test_path | awk '{ print $2 }')
echo "test_sha = $test_sha"
)
fi
nixfile=default.nix
#echo "writing nixfile head"
cat >$nixfile <<EOF
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
EOF
if $use_git
then
cat >>$nixfile <<EOF
small = fetchgit {
url = "git://$git_host:$git_port/$git_path";
sha256 = "$git_sha";
name = "test-$version-small";
};
EOF
elif $use_http
then
cat >>$nixfile <<EOF
small = fetchurl {
TODO
};
EOF
else
cat >>$nixfile <<EOF
small = writeText "test-$version-small" "version=$version";
EOF
fi
cat >>$nixfile <<EOF
in
# large
stdenv.mkDerivation {
name = "test-$version";
#buildInputs = [ small ];
phases = [ "buildPhase" ];
buildPhase = ''
mkdir \$out
cp -r \${small} \$out/small
env >\$out/env.txt
'';
EOF
echo "writing nixfile body ..."
# generate the "large" expression
for (( i=0; i<nix_body_size; i++ ))
do
echo " ENV_$i = \"$i\";"
done >>$nixfile
echo "writing nixfile body done"
#echo "writing nixfile tail"
cat >>$nixfile <<'EOF'
}
EOF
echo "running nix-build"
time nix-build || true # allow to fail
#echo "hit enter to stop servers"; read
if $use_git
then
git_pid=$(cat $git_pid_file)
echo stopping git server $git_pid
kill $git_pid
rm $git_pid_file
fi
if $use_http
then
echo stopping http server
kill $http_pid
fi
default.nix example with nix_body_size=10
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
small = fetchgit {
url = "git://127.0.0.1:8120/test";
sha256 = "sha256-EPf/MJ9AC1zE91VaNU77XuqODdCmBBFblNFq4ZkIl10=";
name = "test-2022-10-11.11-14-50-small";
};
in
# large
stdenv.mkDerivation {
name = "test-2022-10-11.11-14-50";
#buildInputs = [ small ];
phases = [ "buildPhase" ];
buildPhase = ''
mkdir $out
cp -r ${small} $out/small
env >$out/env.txt
'';
ENV_0 = "0";
ENV_1 = "1";
ENV_2 = "2";
ENV_3 = "3";
ENV_4 = "4";
ENV_5 = "5";
ENV_6 = "6";
ENV_7 = "7";
ENV_8 = "8";
ENV_9 = "9";
}
maybe add more buildInputs to large
I think you misread my comment. Can you repro if large is the input to small?
aah, sorry ... in my experience, input is small
no repro with nix_body_size=0
I tried up to a 20GB derivation without triggering the error.
It seems this is coming up in CI a lot. Is there a single-user/multi-user distinction here?
Maybe it requires a combination of heavy load and large derivation to trigger, what about limiting the cpu usage with cgroups and try again?
here's three failures since I added logging:
==> nixbuild-1665485511-SvXJzTCV <==
building '/nix/store/mk8vga75y220cv3kakqy4bdl6zh28bif-unit-script-nixos-activation-start.drv'...
building '/nix/store/9njc1nw481zyszmy04kgdw394yswfqxq-reload-container.drv'...
building '/nix/store/30vwv57m5zngh7rq34iqskfqknb57l0f-unit-mount-pstore.service.drv'...
building '/nix/store/r048mhjnx2dna72bkisay7rawldhshjp-unit-container-.service.drv'...
building '/nix/store/k0nf7hv80z059r2kwcvj5qzx671lzf9i-unit-nixos-activation.service.drv'...
building '/nix/store/2lgv1wrncf3flnxhnw5hswdv2j2zz7lr-openconnect-9.01.drv'...
building '/nix/store/kmqk1x7a85ls2a87c38czhmjv5701h6n-fix-paths.patch.drv'...
building '/nix/store/fjdb602kr5hg5wj027x1lgrab624hm7w-networkmanager-1.40.0.drv'...
building '/nix/store/l1b69i7lfiw8cxwgybnkqclq86rdcni5-networkmanager-aarch64-unknown-linux-gnu-1.40.0.drv'...
error: derivation '/nix/store/g1ir4bxvfz503zpl00py69h30by0dksr-hwdb.bin.drv' requires non-existent output 'out' from input derivation '/nix/store/l1b69i7lfiw8cxwgybnkqclq86rdcni5-networkmanager-aarch64-unknown-linux-gnu-1.40.0.drv'
==> nixbuild-1665498596-0NxwvjdR <==
/nix/store/m2j8lj8d16k2dcxyfd3lsdnz98bahxxb-phodav-2.5
copying 0 paths...
copying path '/nix/store/3j3ma11xs8ly6865m8fprw75pvj1vc0d-iwd' from 'https://cache.nixos.org'...
copying path '/nix/store/d80iax5ixj7byx0088svm8jal9hphn42-spice-protocol-0.14.4' from 'https://cache.nixos.org'...
copying path '/nix/store/m2j8lj8d16k2dcxyfd3lsdnz98bahxxb-phodav-2.5' from 'https://cache.nixos.org'...
building '/nix/store/hnhx94ynlska8zkflcpfmqcrrm7asq10-usbredir-0.12.0.drv'...
copying path '/nix/store/h2381f1k78d30kylapkk3p86kj9wir1z-phodav-2.5-dev' from 'https://cache.nixos.org'...
building '/nix/store/x7bl8zsnl014yfmk58kdg0wcp90k2v3i-iwd-1.29.drv'...
building '/nix/store/x8sr8lyldmz9vv6j57hix61av5zcw9w2-spice-gtk-0.41.drv'...
error: derivation '/nix/store/c4xc8lmaxgykfwsnjrgalgcf9mfkg2lv-system-path.drv' requires non-existent output 'out' from input derivation '/nix/store/x7bl8zsnl014yfmk58kdg0wcp90k2v3i-iwd-1.29.drv'
==> nixbuild-1665508919-lW97k6ID <==
building '/nix/store/f1ikprcgwzwk0wcbwmj1aa9hpy20zrd5-unit-home-assistant.service.drv'...
building '/nix/store/c3s7gvc99dd9j8vxp2akph9643si5ci1-system-units.drv'...
building '/nix/store/nis6lpl25hb3ava21d4cln73hi60189i-system-units.drv'...
building '/nix/store/87si9hfa4dk2nx8mkwavxy0kbi6yp9zf-etc.drv'...
building '/nix/store/a3504wv0qwx8qnr6a07k278730myd7j8-etc.drv'...
building '/nix/store/fn2qz2cmayw8204sxyryy59ls9awkrlz-nixos-system-xeep-22.11.20221011.b78e482.drv'...
building '/nix/store/mfl8v7mlh9p2s419rxr8xyr05ybb3zvx-nixos-system-xeep-22.11.20221011.b78e482.drv'...
building '/nix/store/yvybpxh94a81wb77fq8q72ixjgc4mxs3-nixos-system-xeep-22.11.20221011.b78e482-to-cached.drv'...
building '/nix/store/28b81hvvkf4324yxfvnhzzi7s5ply1rx-udisks-2.9.4.drv'...
error: derivation '/nix/store/50v7swmbdr0gka6z38f4ng20fkqcrpky-system-path.drv' requires non-existent output 'out' from input derivation '/nix/store/28b81hvvkf4324yxfvnhzzi7s5ply1rx-udisks-2.9.4.drv'
@tomberek it's that after that that PR, instead of just using the DB for this stuff (and reading drv files) it is doing a bunch of in-memory maps building and looking up. Very easy to have concurrency bugs with this stuff. I am expecting that more than a blatant logic bug like forgetting the map may not be total.
Not sure if relevant, but I see this fairly consistently (i.e. almost every time I nixos-rebuild), mostly with man outputs:
error: derivation '/nix/store/zbbh13kgkzi2xjapiax8l5vzkr85izli-user-environment.drv' requires non-existent output 'man' from input derivation '/nix/store/3agcqdrkikqff86777hq829880qjv673-i686-w64-mingw32-stage-final-gcc-wrapper-11.3.0.drv'
Not sure if relevant, but I see this fairly consistently (i.e. almost every time I
nixos-rebuild), mostly withmanoutputs:error: derivation '/nix/store/zbbh13kgkzi2xjapiax8l5vzkr85izli-user-environment.drv' requires non-existent output 'man' from input derivation '/nix/store/3agcqdrkikqff86777hq829880qjv673-i686-w64-mingw32-stage-final-gcc-wrapper-11.3.0.drv'
get a log with nixos-rebuild switch -vvvvv maybe we'll get some extra info
I dropped a -v (only had 4), but managed to reproduce it:
https://cadance.vs49688.net/cache/badlog.redacted.txt.xz (32MB uncompressed)
EDIT: It always seems to happen on i686-w64-mingw32-stage-final-gcc-wrapper-11.3.0.drv, which is one of:
(pkgsCross.mingwW64.stdenv.cc.override({
extraBuildCommands = ''
printf '%s' '-L${pkgsCross.mingwW64.windows.mcfgthreads}/lib' >> $out/nix-support/cc-ldflags
printf '%s' '-I${pkgsCross.mingwW64.windows.mcfgthreads.dev}/include' >> $out/nix-support/cc-cflags
'';
}))
(pkgsCross.mingw32.stdenv.cc.override({
extraBuildCommands = ''
printf '%s' '-L${pkgsCross.mingw32.windows.mcfgthreads}/lib' >> $out/nix-support/cc-ldflags
printf '%s' '-I${pkgsCross.mingw32.windows.mcfgthreads.dev}/include' >> $out/nix-support/cc-cflags
'';
}))
(how does printf '%s' '-I${pkgsCross.mingw32.windows.mcfgthreads.dev}/include' >> $out/nix-support/cc-cflags work without adding whitespace before the new flag?)
Sheer luck, there happens to already be a trailing whitespace already there. I copied it from https://github.com/NixOS/nixpkgs/issues/156343#issuecomment-1019557774
This is a failed nix build -vvvvv, it's a big large: fail.log:
error: derivation '/nix/store/xhjazx61n5ylj5js21mjsvkq268yxrzb-hwdb.bin.drv' requires non-existent output 'out' from input derivation '/nix/store/7d8w142a8akv2789ks551m11pps5mzja-udisks-2.9.4.drv'