bug: Duplicate entries when using builtins.readFile

[supermarin]

Is there an existing issue for this?

• [X] I have searched the existing issues

Issue description

I've just noticed that all the files sourced with builtins.readFile end up having duplicated contents, like it was sourced twice. Tested with .ssh/config, neovim config, .sqliterc, etc. The issue persists across any dotfile that's installed like this:

  home.file.".sqliterc".text = builtins.readFile ./sqliterc;

The resulting file

.headers on
.mode column

.headers on
.mode column

Here's contents of the source, result file and proof that ~/.sqliterc is indeed installed by home-manager:

$ cat dotfiles/sqliterc
.headers on
.mode column
$ cat ~/.sqliterc
.headers on
.mode column

.headers on
.mode column
$ ls -la ~/.sqliterc
lrwxr-xr-x 1 supermarin staff 72 Jun 28 11:25 /Users/supermarin/.sqliterc -> /nix/store/1g5zzz7fcai00dc068w0q75d87mraxig-home-manager-files/.sqliterc

Maintainer CC

No response

System information

Flake based install.
 - system: `0`
 - host os: `Darwin 21.4.0, macOS 12.3.1`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.9.1`
 - channels(root): `"nixpkgs"`
 - channels(supermarin): `""`

...
From flake.lock

    "home-manager": {
      "inputs": {
        "nixpkgs": [
          "nixpkgs"
        ],
        "utils": "utils"
      },
      "locked": {
        "lastModified": 1656367977,
        "narHash": "sha256-0hV17V9Up9pnAtPJ+787FhrsPnawxoTPA/VxgjRMrjc=",
        "owner": "nix-community",
        "repo": "home-manager",
        "rev": "3bf16c0fd141c28312be52945d1543f9ce557bb1",
        "type": "github"
      },
      "original": {
        "owner": "nix-community",
        "repo": "home-manager",
        "type": "github"
      }
    },

EDIT: using nix-darwin as well, so the system is rebuilt with darwin-rebuild --flake .#macos switch. Home-manager is used as a module.

[supermarin]

Tested on nixOS 22.05, the issue isn't there. Will close this and open in nix-darwin

[supermarin]

Reopening this on home-manager side, see https://github.com/LnL7/nix-darwin/issues/475#issuecomment-1230788640 for a good explanation of what's happening.

[stale[bot]]

Thank you for your contribution! I marked this issue as stale due to inactivity. Please be considerate of people watching this issue and receiving notifications before commenting 'I have this issue too'. We welcome additional information that will help resolve this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

• If this is resolved, please consider closing it so that the maintainers know not to focus on this.
• If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
• If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

• If you are also experiencing this issue, please add details of your situation to help with the debugging process.
• If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

Don't be afraid to manually close an issue, even if it holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen – nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.

[supermarin]

Not stale, should be addressed

[rycee]

Hmm, from I can tell from the description in the nix-darwin issue you effectively have something like

home.file.testfile = lib.mkMerge [
  { text = builtins.readFile ./testfile; }
  { text = builtins.readFile ./testfile; }
];

If that is correct then I think the expected behavior would be to have the content of ./testfile twice in ~/testfile.

[rycee]

You may want to try something like

home.file.".sqliterc".source = ./sqliterc;

instead.

[ncfavier]

The fact that imported modules cannot be deduplicated is a known issue with the module system. Nothing Home Manager can fix, so I'll close this issue.

In general you should prefer referring to modules by path, i.e.

home-manager.users.supermarin = ../home.nix;

See https://github.com/NixOS/nix/issues/6866, https://github.com/NixOS/nix/pull/7356 for related discussions.

[supermarin]

@ncfavier not sure if we're on the same page - I did import home.nix exactly in that way: https://github.com/LnL7/nix-darwin/issues/475#issuecomment-1211395821

[ncfavier]

Nope:

home-manager.users.supermarin = import ../home.nix;

vs.

home-manager.users.supermarin = ../home.nix;

If you import, the module system doesn't know what the module is called, so it can't deduplicate it.

[supermarin]

Gotcha. I've switched to a different config since:

  home-manager.useGlobalPkgs = true;
  home-manager.useUserPackages = true;
  home-manager.users.supermarin.imports = [
    ../home.nix
    ../home-services.nix
    ../mail.nix
  ];

Just tested this in both places and I can't reproduce the issue. Switched the configuration.nix one to the old style: home-manager.users.supermarin = import ../home.nix; and I'm getting an error below (which I think is good):

error: The option `home-manager.users.supermarin.programs.git.package' is defined multiple times.

       Definition values:
       - In `/nix/store/r549gfmgan44da7q3rgl449vfr7mawd3-source/home.nix': <derivation git-with-svn-2.38.1>
       - In `/nix/store/r549gfmgan44da7q3rgl449vfr7mawd3-source/nixos/configuration.nix': <derivation git-with-svn-2.38.1>
(use '--show-trace' to show detailed location information)

If I use your syntax in both places, the build succeeds and still can't reproduce the problem, as in there's no duplicated entries. Let me dig in a bit more.

[supermarin]

Nope, can't reproduce anymore. Either I'm holding something wrong, or this was fixed recently in nix. So to sum up:

• either build succeeds and files contents aren't duplicated
• or build fails warning of a duplicate definition (which is my preferred outcome)

EDIT: tested @rycee 's suggestion .source = vs .text = builtins.readFile as well and it doesn't make a difference.

[berbiche]

Nope:

home-manager.users.supermarin = import ../home.nix;

vs.

home-manager.users.supermarin = ../home.nix;

If you import, the module system doesn't know what the module is called, so it can't deduplicate it.

I believe you can use _file to set a module's name if generating nix modules but I've never encountered this situation