bug: bin/home-manager-generation fails due to conflicts with prior installed nix profile entries

[typetetris]

Is there an existing issue for this?

• [X] I have searched the existing issues

Issue description

Building activationPackage and calling bin/home-manager-generation produces an error about conflicting packages in the user environment managed via nix profile. The conflicting package comes from a prior invocation of some bin/home-manager-generation produced by the same version of home-manager.

nix profile list output:

0 - - /nix/store/cixwgxv2fsibh4gp183nkqk3w7yhac59-home-manager-path

bin/home-manager-generation output (after removing one entry from home.packages):

Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
error: packages '/nix/store/hf1zyw9rpc2vwsljyyvx0k2g7cilaaqc-home-manager-path/opt/teams/locales/am.pak' and '/nix/store/7il2bcjnh2zksfsi67ad7rdqisb4cird-teams-1.4.00.26453/opt/teams/locales/am.pak' have the same priority 5; use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' to change the priority of one of the conflicting packages (0 being the highest priority)

Oops, Nix failed to install your new Home Manager profile!

Perhaps there is a conflict with a package that was installed using
"nix profile install"? Try running

    nix profile list

and if there is a conflicting package you can remove it with

    nix profile remove {number | store path}

Then try activating your Home Manager configuration again.

Maintainer CC

@rycee @blaggacao @FlorianFranzen

System information

- system: `"x86_64-linux"`
 - host os: `Linux 5.15.29, NixOS, 22.05 (Quokka), 22.05.20220323.1d08ea2`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.7.0`
 - channels(root): `"nixos-21.05.2693.d5aadbefd65, nixos-20.09-20.09.4393.068984c00e0"`
 - nixpkgs: `/etc/nixpkgs/channels/nixpkgs`

[ratsclub]

Removing all profiles with nix profile remove {number} solved the issue for me temporarily but I need to do this for every switch.

Using this revision on my flake.lock fixes this completely:

      "locked": {
        "lastModified": 1648751066,
        "narHash": "sha256-pYUSID9rSgYnl4PNa45/haCaHUKzY+Ul0fkqqSGflxs=",
        "owner": "nix-community",
        "repo": "home-manager",
        "rev": "cf62e96bf7c72e6a88e0bd43165110f42e44cdb4",
        "type": "github"
      },

[sullyj3]

I think this is related to https://github.com/NixOS/nix/issues/6143

[typetetris]

nix profile list only lists a number and an store path for the home manager stuff. How should it know, wether its another "version" of the "same" package?

[wrvsrx]

I think that's because that home-manager is incompatible with nix profile (https://nix-community.github.io/home-manager/index.html#sec-install-standalone). Once you modify you profile by nix profile, such as nix profile install xxx, home-manager can't update your profile correctly.

I met the same problem today, and I solve that by delete profiles operated by nix profile and use home-manager to recreate profile.

Besides, that should not be relevant to home-manager version. @ratsclub 's method is ineffective, at least for me

[sullyj3]

nix profile support was added in #2840

[YorikSar]

I can confirm that it doesn't work for me on macOS as well. It seems nix profile install doesn't know about the fact that home-manager-path should be replaced, and tries to add a new one. If I do nix profile install /nix/store/...-home-manager-path manually and run home-manager switch again, I get 2 lines in the profile:

1 - - /nix/store/32ik3vjz5lfs9w68616c8fjwgkphlgb1-home-manager-path
2 - - /nix/store/32ik3vjz5lfs9w68616c8fjwgkphlgb1-home-manager-path

I would guess that this happens because home-manager looses information about flake from which the path is generated. Maybe we should install it as nix profile install .#homeConfigurations.${USER}.config.home.path instead of just ${cfg.path} in https://github.com/nix-community/home-manager/blob/8d38ca886880265d523a66fe3da4d42e92ab0748/modules/home-environment.nix#L577?

Note that judging by the comment, if home-manager is used from NixOS or nix-darwin, then it properly removes itself from the profile if home-manager-path is installed externally: https://github.com/nix-community/home-manager/blob/8d38ca886880265d523a66fe3da4d42e92ab0748/modules/home-environment.nix#L574-L576

[yorickvP]

I'm using this flake output as a workaround:

{
      apps.x86_64-linux.update-home = {
        type = "app";
        program = (self.legacyPackages.x86_64-linux.writeScript "update-home" ''
          set -euo pipefail
          old_profile=$(nix profile list | grep home-manager-path | head -n1 | awk '{print $4}')
          echo $old_profile
          nix profile remove $old_profile
          ${self.packages.x86_64-linux.yorick-home}/activate || (echo "restoring old profile"; ${self.legacyPackages.x86_64-linux.nix}/bin/nix profile install $old_profile)
        '').outPath;
      };
}

[blaggacao]

this might be solved by https://github.com/NixOS/nix/pull/6522

[YorikSar]

@blaggacao i don't think so. That PR allows to add new derivation to the profile with a priority. Here we need to somehow replace old home-manager-path in profile with the new one.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/depreciate-the-use-of-nix-env-to-install-packages/20139/32

[RonnyPfannschmidt]

my current impression is, that it seems necessary to use a nix profile command to install the switched version properly

[Congee]

Hi, @yorickvP would you mind posting your while flake.nix? I cannot figure out how to apply your snippet into my flake.nix. Thanks!

[Congee]

Never mind, self is context dependent, and my workaround script looks like

{
  description = "";
  inputs = { };

  outputs = { self, home-manager, darwin, nixpkgs, nixos, ... } @ inputs: {
    # home-manager
    homeConfigurations = {
      desktop = home-manager.lib.homeManagerConfiguration ({ });
    };

    desktop = self.homeConfigurations.desktop.activationPackage;
    apps.x86_64-linux.update-home = {
      type = "app";
      program = (nixpkgs.legacyPackages.x86_64-linux.writeScript "update-home" ''
        set -euo pipefail
        old_profile=$(nix profile list | grep home-manager-path | head -n1 | awk '{print $4}')
        echo $old_profile
        nix profile remove $old_profile
        ${self.desktop}/activate || (echo "restoring old profile"; ${nixpkgs.legacyPackages.x86_64-linux.nix}/bin/nix profile install $old_profile)
      '').outPath;
    };
  };
}

[sbaildon]

Trying to follow along to understand what's my issue and what's an issue with home-manager at the moment,

After

$ nix build path:.#homeConfigurations.sbaildon.activationPackage
$ result/activate

Everything works okay and I have one nix profile

$ nix profile list
0 - - /nix/store/5625nz68pg65c9i57w382g87j0pi1n8f-home-manager-path

If update home.nix, one of the modules referenced by flake.nix, add an entry to packages[], then try to update, I get this

$ home-manager switch --flake 'path:/Users/sbaildon/.config/nixpkgs#sbaildon'
error: files '/nix/store/k3w9lfkimp8vh6a4fcsblnqagdn66869-home-manager-path/bin/man' and '/nix/store/b2aa3vjj6rz3d76hjbbvpkmrja4mphj7-home-manager-path/bin/man' have the same priority 5;

Is this the current known bad behaviour?

flake.nix

{
  description = "home";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/7d9f27f92df31b59cac7b922a7a4e7ca07cca428";
    home-manager = {
      url = "github:nix-community/home-manager/c1addfdad3825f75a66f8d73ec7d2f68c78ba6f8";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { nixpkgs, home-manager, ... }:
    let
      system = "x86_64-darwin";
      pkgs = nixpkgs.legacyPackages.${system};
    in {
      homeConfigurations.sbaildon = home-manager.lib.homeManagerConfiguration {
        inherit pkgs;

        modules = [
          ./home.nix
	  { 
            programs.home-manager.enable = true;
            home = {
              stateVersion = "22.11";
              homeDirectory = "/Users/sbaildon";
              username = "sbaildon";
            };
          }
	];
      };
    };
}

[Minion3665]

Trying to follow along to understand what's my issue and what's an issue with home-manager at the moment,

After

$ nix build path:.#homeConfigurations.sbaildon.activationPackage
$ result/activate

Everything works okay and I have one nix profile

$ nix profile list
0 - - /nix/store/5625nz68pg65c9i57w382g87j0pi1n8f-home-manager-path

If update home.nix, one of the modules referenced by flake.nix, add an entry to packages[], then try to update, I get this

$ home-manager switch --flake 'path:/Users/sbaildon/.config/nixpkgs#sbaildon'
error: files '/nix/store/k3w9lfkimp8vh6a4fcsblnqagdn66869-home-manager-path/bin/man' and '/nix/store/b2aa3vjj6rz3d76hjbbvpkmrja4mphj7-home-manager-path/bin/man' have the same priority 5;

Is this the current known bad behaviour?

flake.nix

Yep, that looks like it

nix profile list \
  | { grep 'home-manager-path$' || test $? = 1; } \
  | awk -F ' ' '{ print $4 }' \
  | cut -d ' ' -f 4 \
  | xargs -t $DRY_RUN_CMD nix profile remove $VERBOSE_ARG
# ^ Remove old profiles; from https://github.com/nix-community/home-manager/blob/8d38ca886880265d523a66fe3da4d42e92ab0748/modules/home-environment.nix#L582

You can use something like this command before activating home manager as a workaround to remove the old profiles

[sbaildon]

I was sure to pin a version of home-manager that included nix path support, found in the PR https://github.com/nix-community/home-manager/pull/2840/files, but it didn't work for me.

I blew away my entire /nix for unrelated reasons and started fresh, turns out everything works perfectly. home-manager switch --flake <path> has no problems; I'm using commit c1addfd. Maybe I had a messed up path or something, but I'm not sure.

[sullyj3]

I'm currently getting it sometimes. Sometimes home-manager switch --flake <path> works fine, sometimes it doesn't and it's not clear to me what the difference is.

[alexshpilkin]

It seems to me as though there was some sort of misunderstanding as to how nix profile works when the initial support was implemented in https://github.com/nix-community/home-manager/pull/2840. Specifically, nix profile install (unlike, AFAICT, nix-env -i, though I’ve hever used it) is not in the business of replacing old versions of packages when new ones are installed: nix profile install $installable; nix profile install $installable will net you two entries for $installable in your profile (and if $installable is not an immutable reference and you wait a while between the commands, the two copies might well be different and most probably cause a conflict). If you use a mutable flake reference, you can nix profile upgrade $installable, but AFAIK a store path like HM uses (?) must be nix profile removed first before nix profile installing the new version, though that will unfortunately count as creating two generations of the profile in question.

(Why does it work that way? I don’t really know, but the problem seems to be that there is really no way to tell if installables are “the same” in some Platonic sense; nix-env can use package names, but a flake installable’s identity is its URL—which might well be different for two versions a user would consider to be “the same”—and its output name—which might be “default”, “static-debug”, or something similarly non-descriptive for a flake that only packages a single piece of software. So it doesn’t try.)

Nowhere in the nix profile support code in home-environment.nix do I see any attempt to remove the old generation before installing the new one. And indeed, every time home-manager switch fails, I see that the old version of home-manager-path has not been removed from my default profile (and removing it first fixes the problem); every time it succeeds, I see the new version together with one or several old ones installed simulaneously (apparently, if I only added some new packages, there might not be a conflict).

[sullyj3]

So is it just a matter of home-manager needing to automatically remove previously installed profiles? I guess creating two each time isn't ideal, but it seems tolerable, especially if there's not a clear better solution.

[alexshpilkin]

@sullyj3 As far as I’m able to tell, yes, though if implemented as two separate commands nix profile history will look like

Version N-2:
    home-manager-path: <install previous generation>

Version N-1:
    home-manager-path: <remove previous generation>

Version N:
    home-manager-path: <install current generation>

which seems less than perfect, especially if somebody rolls back to N−1 without realizing they may not have any accessible home-manager at all until they roll back further to N−2. As I’ve said above, there is an atomic remove-and-install command in the form of nix profile upgrade, but it’s limited and doesn’t work at all with things installed by store path. So if one wants to deal with this wrinkle (which is defintely not a requirement for a fix), I see two possibilities:

• Extend nix profile upgrade or otherwise teach Nix to atomically replace one user-specified store path in a profile with another;
• Change the way HM activation works so that the existing Nix functionality is sufficient: for example, make the build result itself be a flake, nix profile install home-manager-path --override-flake home-manager-path $pathtothatflake to install and nix profile upgrade $THATENTRY --override-flake home-manager-path $newpathtothatflake to switch (the $THATENTRY part is a problem—all the ways to specify the target to nix profile upgrade suck in one way or another, that needs to be fixed in Nix).

[sullyj3]

It feels like maybe there's a lack of impetus to fix this due to flakes' nominally experimental status, despite their de facto widespread usage.

[arianvp]

Instead of using the imperative nix profile commands home-manager should use nix build --profile <name here> to declaratively and atomically switch the profile from one activation package to another. This should fix this problem

[RonnyPfannschmidt]

@arianvp how would that work in practice? The activation command runs the switch and does not have the nix expression /flake avaliable

[RonnyPfannschmidt]

@arianvp current home manager contributes to the user profile instead of replacing it, so taking control of the full profile seems like a "breaking" change at first

[sullyj3]

ref https://github.com/NixOS/nix/issues/6349 and https://github.com/NixOS/nix/issues/7054

[ameyp]

As far as I can tell, this isn't actually fixed. I'm still seeing the same error with a fresh install of home-manager.

@typetetris Is it fixed for you?

[alexshpilkin]

@ameyp It is fixed for my existing installation (of the master branch), at least, and like the patch implies, I’m seeing two profile generations created per invocation of home-manager switch (the first to remove the old store path, the second to add the new one).

[ameyp]

I'm installing home-manager on a non-nixOS system with:

nix-channel --add https://github.com/nix-community/home-manager/archive/master.tar.gz home-manager
nix-channel --update

export NIX_PATH=$HOME/.nix-defexpr/channels:/nix/var/nix/profiles/per-user/root/channels${NIX_PATH:+:$NIX_PATH}
nix-shell '<home-manager>' -A install

When I run home-manager switch, I get

ubuntu% home-manager switch                  
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
/nix/store/0n7rlf3rlbzafj5jf9gjdznm0wh6i174-home-manager-generation
Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
nix profile remove /nix/store/4ivzn5r7p87abhycb3jvaxzirlqb9m3m-home-manager-path
removing 'home-manager-path'
error: files '/nix/store/xdlpraypxdimjyfrr4k06narrv8nmfgh-nix-2.11.1/share/zsh/site-functions/_nix' and '/nix/store/5w533q5gfzkm2wh26v8v7m8byvr16nry-home-manager-path/share/zsh/site-functions/_nix' have the same priority 5; use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' or type 'nix profile install --help' if using 'nix profile' to find out howto change the priority of one of the conflicting packages (0 being the highest priority)

Oops, Nix failed to install your new Home Manager profile!

Perhaps there is a conflict with a package that was installed using
"nix profile install"? Try running

    nix profile list

and if there is a conflicting package you can remove it with

    nix profile remove {number | store path}

Then try activating your Home Manager configuration again.

nix profile list gives:

0 - - /nix/store/xdlpraypxdimjyfrr4k06narrv8nmfgh-nix-2.11.1

I'm specifically encountering this when I have a programs.zsh entry in my home.nix. The problem persists even if I remove /nix, $HOME/.nix* and reinstall nix from scratch.

[ameyp]

Perhaps the root cause of https://github.com/nix-community/home-manager/issues/2995 is not the same as the root cause of this bug? If so, my bad for conflating the two.

[alexshpilkin]

@ameyp Right, I don’t know if this counts as a bug (it certainly seems undesirable), but in any case that’s a different problem: this issue is about different versions of the home-manager–managed environment conflicting among themselves inside a new-style profile, but you are seeing a single (fresh) version of said environment (the home-manager-path in your log) conflicting with a nix that’s been installed there separately and manually. To me this does indeed look like a new-style nix profile version of #2995, which has probably been wrongfully closed. (While with nix-env you could circumvent that problem by setting priorities, I don’t think nix profile allows that?)