nixos: permanently broken install due to bad cache public key

[Pandademic]

Describe the bug I'm not sure this is the right repo to submit this at, or if this is a bug or a documentation omit-ion but:

I was migrating my nixos config to a flake, and added the cachix repo for Hyprland as per these instructions.

Unfortunately, I manually typed the public key part, and messed up.Since I didn't need to rebuild hyprland when I rebuilt the system, I was able to rebuild successfully.However, after running nix-collect-garbage -d to do a clean rebuild and test, I cannot rebuild my system, nor run any nix command, since there is an invalid public key and so it cannot use the cachix substituter, and fails.I have tried editing my config and rebuilding, which obviously failed due to not being able to even get the nixpkgs input of the flake, and editing /etc/nixos/nix.conf which doesnt work due to being on a read only fs. As far as I can tell my system is gone, the only solution being re installing nixos, which is undesirable.

This seems to be a bug, as this is a rather easy mistake to make for the only solution being re installing.

Steps To Reproduce

1. add a substituter to nix.settings
2. mess up its public key

Expected behavior you cannot rebuild or use nix

nix-env --version output nix-env (Nix) 2.12.0

(nix-env works because it does not download anything`

[bjornfor]

[..] and editing /etc/nixos/nix.conf which doesnt work due to being on a read only fs. As far as I can tell my system is gone, the only solution being re installing nixos [..]

A way out:

$ sudo mv /etc/nix/nix.conf /etc/nix/nix.conf.bak
$ sudo cp /etc/nix/nix.conf.bak /etc/nix/nix.conf
$ sudo chmod +w /etc/nix/nix.conf
$ sudo $EDITOR /etc/nix/nix.conf

[Pandademic]

@bjornfor, thank you so much!

I did not use your comment,however, I saw it too late.

I seem to have fixed it, by rebooting with my pre-flake config(still stored in /etc/nixos). Everything worked perfectly there, so I cd'd into my nix flakes folder, and rebuilt from the old config.Then, after rebooting with the just created version, I did a home manager switch, and everything worked fine again.( in the new flake config, I deleted the bad key).

Should I close this issue?

[Pandademic]

Also, is this a bug or not?

We seem to have 2 workarounds to whatever it is, anyway.

[bjornfor]

I'll leave it up to Nix devs to decide whether it's a bug or not.

[Et7f3]

being on a read only fs

To do in extreme case: remount as rw mount -o rw,remount /nix/store

[Pandademic]

That works too, do you think this is a bug?

[Et7f3]

Have you the config snippet that didn't work ? Were you modifying trusted-public-keys if yes it seems more a nixpkgs bug than nix bug: you say nix to don't trust a key and it respected what you asked.

if you don't remember config nor able to reproduce I fear that we can't fix with the infos you gave.

[Pandademic]

hey @Et7f3.

Have you the config snippet that didn't work ?

The exact snippet that didn't work to fix it or the exact snippet that caused the trouble? ( I have both)

... you say nix to don't trust a key and it respected what you asked.

The issue is not with nix not trusting the public key, but rather shutting down after encountering a bad one.

... seems more a nixpkgs bug than nix bug

I didn't know where to report this, as it could very well be considered a bug in one of either:

• something in nixpkgs/nixos's nix.settings
• a nix bug with nix having no mechanism to modify its (broken) settings

[Pandademic]

btw @Et7f3 if you prefer speaking french let me know.

[AnidemDex]

Can confirm that doing https://github.com/NixOS/nix/issues/8271#issuecomment-1527846006 can be helpful in case you have wrong trusted public keys. Somehow I added a wrong public key from hyprlen and nixos-build refused to update the configuration so I have to remove that key manually from nix.conf

[ChanceHarrison]

A user in the #nix.nixos.org Matrix channel (direct link to message) recently ran into this same issue. We were stuck until we found this issue. I just wanted to use this as an opportunity to bump this issue if nothing else but to indicate that this is still negatively impacting (new) users.

In addition to any potential fixes (whether those would be in the NixOS module for nix.settings or in Nix itself as @Pandademic indicated or elsewhere), it would be good if this issue and workaround was more discoverable. Adding a section/note to https://nixos.wiki/wiki/Binary_Cache might be ideal. [Edit: Done!] ~~I will see if I can get around to it and update this comment if I do.~~

[Pandademic]

@ChanceHarrison thank you for the update to the wiki page! (sorry for late reply, this account is not checked much)

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/help-nixos-kde-plasma-6-kept-crashing-and-freezing-even-in-tty/58425/22

[pigsinablanket]

Another workaround is to manually pass in the options substituters and trusted-public-keys to override the values in /etc/nix/nix.conf

nixos-rebuild switch --option substituters "https://cache.nixos.org/" --option trusted-public-keys "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="

[damien-rezdy]

Another workaround is to manually pass in the options substituters and trusted-public-keys to override the values in /etc/nix/nix.conf

nixos-rebuild switch --option substituters "https://cache.nixos.org/" --option trusted-public-keys "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="

Thank you, this one did the trick for me.