nix The nix (Nix) 2.21.0pre20240222_6a5210f statically compiled is not able to install packages in profile

Describe the bug

The command nix profile install nixpkgs#hello is not able to install the hello package (or any other) for
nix (Nix) 2.21.0pre20240222_6a5210f statically compiled if /nix does not exist and is not possible to be created.

It errors:

warning: '/nix/var/nix' does not exist, so Nix will use '/home/abcuser/.local/share/nix/root' as a chroot store
error: opening directory '/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1': No such file or directory
bash: line 5: hello: command not found

Since nix 2.10.0 it should work:

On Linux, if /nix doesn’t exist and cannot be created and you’re not running as root, Nix will automatically use ~/.local/share/nix/root as a chroot store. This enables non-root users to download the statically linked Nix binary and have it work out of the box. From: https://discourse.nixos.org/t/nix-2-10-0-released/20291
The Nix/NixOS release team: What's new in Nix 2.8.0 - 2.12.0?, start=462&end=546
Matthew Croughan - What Nix Can Do (Docker Can't) - SCaLE 20x, start=4282&end=4389, it is from March 9-12, 2023
[...] We've also been looking at the statically built version of Nix which is good for "single user" / home-directory-managed installs of Nix. It's a great idea and simplifies the onboarding a good bit. [...] https://news.ycombinator.com/item?id=34957953

Well, it is an "old problem":

2015: How do I install Nix on a custom directory?
2015: https://github.com/NixOS/nix/issues/512#issuecomment-225517822
2017: Nixpkgs Overlays – A place for all excluded packages by Nicolas B. Pierron (NixCon 2017), start=467 end=501. What would be "the correct way"?
2018: Recently I had to install Nix on a university workstation where I couldn't create /nix. As I still wanted to use the official binary cache... Refs.: https://github.com/NixOS/nix/issues/1971
2018: [...] the installer needing to be installed in /nix could be solved by creating a statically linked Nix. By Eelco
2019: Using Nix for Repeatable Python Environments | SciPy 2019 | Daniel Wheeler, start=1712&end=1721
2021: [...] Nix installs stuff into its own area, out of the way of the rest of the system. That area is normally /nix/store. We can’t create that directory out in the root on Glitch. There are various ways you can set things up to work around this https://support.glitch.com/t/install-prebuilt-packages-without-root-from-nixpkgs/43775/1

Steps To Reproduce

It needs podman or docker:

cat > Containerfile << 'EOF'
FROM ubuntu:23.04

RUN apt-get update -y \
 && apt-get install --no-install-recommends --no-install-suggests -y \
     adduser \
     ca-certificates \
     curl \
     file \
 && apt-get -y autoremove \
 && apt-get -y clean \
 && rm -rf /var/lib/apt/lists/*

RUN addgroup abcgroup --gid 4455  \
 && adduser -q \
     --gecos '"An unprivileged user with an group"' \
     --disabled-password \
     --ingroup abcgroup \
     --uid 3322 \
     abcuser

# If is added nix statically compiled works!
# RUN mkdir -pv /nix/var/nix && chmod -v 0777 /nix && chown -Rv abcuser:abcgroup /nix

USER abcuser
WORKDIR /home/abcuser
ENV USER="abcuser"
ENV PATH=/home/abcuser/.nix-profile/bin:/home/abcuser/.local/bin:"$PATH"
ENV NIX_CONFIG="extra-experimental-features = nix-command flakes"

RUN mkdir -pv "$HOME"/.local/bin \
 && cd "$HOME"/.local/bin \
 && curl -L https://hydra.nixos.org/build/250594210/download/2/nix > nix \
 && chmod -v +x nix 
EOF


podman \
build \
--file=Containerfile \
--tag=unprivileged-ubuntu23 .

Running the created image:

podman \
run \
--privileged=true \
--interactive=true \
--tty=true \
--rm=true \
localhost/unprivileged-ubuntu23:latest \
bash \
-c \
'
# Broken
nix profile install nixpkgs#hello
file ~/.nix-profile
hello
'

It prints the following:

warning: '/nix/var/nix' does not exist, so Nix will use '/home/abcuser/.local/share/nix/root' as a chroot store
error: opening directory '/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1': No such file or directory
/home/abcuser/.nix-profile: broken symbolic link to /home/abcuser/.local/state/nix/profiles/profile
bash: line 5: hello: command not found

Expected behavior

The hello package should be installed with no errors.
The user profile in /home/abcuser/.nix-profile should be a symbolic link to /nix/var/nix/profiles/per-user/abcuser/profile.
The hello command should work.

Additional context

Running interactively in the container:

podman \
run \
--interactive=true \
--tty=true \
--rm=true \
localhost/unprivileged-ubuntu23:latest \
bash \
-c \
'
nix flake --version
nix flake metadata nixpkgs
'

Outputs

nix (Nix) 2.21.0pre20240222_6a5210f
warning: '/nix/var/nix' does not exist, so Nix will use '/home/abcuser/.local/share/nix/root' as a chroot store
Resolved URL:  github:NixOS/nixpkgs/nixpkgs-unstable
Locked URL:    github:NixOS/nixpkgs/98b00b6947a9214381112bdb6f89c25498db4959
Description:   A collection of packages for the Nix package manager
Path:          /nix/store/ph5qcvhhkwrcmiz4laabvz9wa6zmy37j-source
Revision:      98b00b6947a9214381112bdb6f89c25498db4959
Last modified: 2024-02-22 01:07:56

Edits

Time passes and new nix versions came out and I have been finding more references to add here, so updating it from time to time.

TODO: write an test that shows it, probably in this file https://github.com/NixOS/nix/blob/master/tests/functional/nix-profile.sh

It started with my self testing it here in nix 2.12.0pre20220829_ddb82ff

nix (Nix) 2.15.0pre20230405_3586e97
nix (Nix) 2.16.0pre20230524_6e45702
nix (Nix) 2.17.0pre20230615_e672d52
nix (Nix) 2.18.0pre20230810_a1fdc68
nix (Nix) 2.19.0pre20231004_2f1c16d
nix (Nix) 2.19.0pre20231110_458e511
nix (Nix) 2.20.0pre20240129_44a0d04
nix (Nix) 2.21.0pre20240222_6a5210f

Other details

How to get latest successful hydra build:

URL=https://hydra.nixos.org/job/nix/master/buildStatic.x86_64-linux/latest
LATEST_ID_OF_NIX_STATIC_HYDRA_SUCCESSFUL_BUILD="$(curl $URL | grep '"https://hydra.nixos.org/build/' | cut -d'/' -f5 | cut -d'"' -f1)"

echo $LATEST_ID_OF_NIX_STATIC_HYDRA_SUCCESSFUL_BUILD

Refs.:

https://github.com/NixOS/nixpkgs/issues/54924#issuecomment-473726288
https://discourse.nixos.org/t/how-to-get-the-latest-unbroken-commit-for-a-broken-package-from-hydra/26354/4

Priorities

Add :+1: to issues you find important.

Aug 30 '22 12:08 PedroRegisPOAR

Some troubleshooting

Note: it does not solve the original issue, in many environments it is not possible to create the /nix.

In one terminal:

Using the built image:

podman \
run \
--name=test-nix-static \
--privileged=true \
--interactive=true \
--tty=true \
--rm=true \
localhost/unprivileged-ubuntu24:latest \
bash

podman \
exec \
--interactive=true \
--tty=true \
--user=0 \
test-nix-static \
bash \
-c \
'
mkdir -p /home/abcuser/.local/share/nix/root/nix
'

In another terminal: 3.

podman \
exec \
--interactive=true \
--tty=true \
--user=0 \
test-nix-static \
bash \
-c \
'
mkdir /nix \
&& mount --bind /home/abcuser/.local/share/nix/root/nix /nix \
&& chown abcuser: /home/abcuser/.local/share/nix/root/nix
'

podman \
exec \
--interactive=true \
--tty=true \
--user=abcuser \
test-nix-static \
bash \
-c \
'
nix profile install nixpkgs#hello
hello
'

Aug 30 '22 14:08 PedroRegisPOAR

paste in your .SHELLrc


nix(){
podman container exists merged_nix;
if [ $? -eq 1 ] ; then
    mkdir -p $HOME/.local/share/nix-aux-dir/{nixwork,nixupper,rootwork,rootupper,etcwork,etcupper}
    podman run --detach-keys="ctrl-d" -it  --name merged_nix\
    -v nix:/nix:O,upperdir=$HOME/.local/share/nix-aux-dir/nixupper,workdir=$HOME/.local/share/nix-aux-dir/nixwork\
    -v nixetc:/etc:O,upperdir=$HOME/.local/share/nix-aux-dir/etcupper,workdir=$HOME/.local/share/nix-aux-dir/etcwork\
    -v nixroot:/root:O,upperdir=$HOME/.local/share/nix-aux-dir/rootupper,workdir=$HOME/.local/share/nix-aux-dir/rootwork\
    --network host --security-opt label=disable --entrypoint sh docker.io/nixos/nix:latest && podman exec -it merged_nix "${@:0}"
else
    podman start merged_nix && podman exec -it merged_nix "${@:0}"
fi
}

first run, dettach using ctrl-d view https://www.youtube.com/watch?v=v62iaHayQP0

Aug 30 '22 17:08 vmath3us

I'm experiencing the same problem with a statically built nix without /nix, however nix shell seems to work:

$ nix profile install nixpkgs#hello
error: opening directory '/nix/store/33l4p0pn0mybmqzaxfkpppyh7vx1c74p-hello-2.12.1': No such file or directory
$ nix shell nixpkgs#hello
$ which hello
/nix/store/33l4p0pn0mybmqzaxfkpppyh7vx1c74p-hello-2.12.1/bin/hello
$ hello
Hello, world!

Dec 21 '22 08:12 wfranzini