flake.nix: provide checks (tests) and docs and related fixes (cont'd)

[ShamrockLee]

Continuation of #3024, which was closed by accident after a failed git rebase. (sorry for the trouble)

Description

This patch provides the Home Manager Module test cases from flake through outputs.checks.${system} and outputs.legacyPackages.${system}. This allows developers to run the tests against the master branch without having to update their environment.

If applied, developers can run the tests by

nix develop --override-input nixpkgs github:NixOS/nixpkgs/nixos-unstable .#tests.run.all

~or simply~

nix flake check --override-input nixpkgs github:NixOS/nixpkgs/nixos-unstable .

Update: As reported by https://github.com/nix-community/home-manager/pull/3024#issuecomment-1156281495 , the multi-platform failure would revive on 22.05 even after the IFD fix. We should probably add the test suite into the checks until https://github.com/NixOS/nix/issues/4265 got fixed.

(nix develop is the Flake analogy of nix-shell --pure -A. The --override-input above makes Nix build against the Nixpkgs nixos-unstable branch instead of the master branch to avoid rebuilding the world.)

Fixes #2722 when running the tests for flake users.

This patch also contains two fixes found when testing the above features:

• Fix impurities when evaluating tests introduced in modules/modules.nix by the appended element pkgsModule that injects _module.args.pkgsPath as <nixpkgs> .
• Get pkgs through inputs.nixpkgs.legacyPackages.${system} whenever available instead of import nixpkgs { inherit "${system}"; }
  • ~This avoids IFD where possible, working around https://github.com/NixOS/nix/issues/4265 on platforms covered by eachDefaultSystem.~ Update: not on NixOS 22.05.

  • This prevents re-evaluation of Nixpkgs. See [1000 instances of nixpkgs](https://discourse.nixos.org/t/1000-instances-of-nixpkgs/17347) for detail.
    

(I don't know why the last sub-bullet is rendered as a code block. It's probably a GitHub bug.

Regarding #3056: This PR does not mean to revert the decisions made in the PR made by rycee. ~Rycee's PR cleared up the hack to use flake.lock content on non-flake system, with the side effect of removing the flake support to access the documents. This PR adds back the flake-based document support plus the tests support, while paying respect to the non-flake infrastructure.~ This PR adds the flake-based test support while keeping the the hard-coded nmd and nmt intact.

Checklist

• [X] Change is backwards compatible. (I think so, but the impurity fix requires experienced people to double check.)

• [X] Code formatted with ./format.

• [X] Code tested through nix-shell --pure tests -A run.all.

  • Tested with both ways mentioned above.

• [ ] Test cases updated/added. See example.

• [ ] Commit messages are formatted like

  {component}: {description}
  
  {long description}
  

  See CONTRIBUTING for more information and recent commit messages for examples.

• If this PR adds a new module

  • [ ] Added myself as module maintainer. See example.

  • [ ] Added myself and the module files to .github/CODEOWNERS.

[ShamrockLee]

To catch up with #1934, I drop those changes related to the nmd and nmt hard-coding (#3056) and move them into https://github.com/ShamrockLee/home-manager/tree/flake-nmd-nmt

They can go into another (draft) PR after this one is merged.

[ShamrockLee]

Cc: @berbiche @ncfavier

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.

[ShamrockLee]

Could anyone take a look at this PR?

Cc: @berbiche

[ShamrockLee]

@stale This is relevent and awaiting merge.

[ncfavier]

Sorry for the delay, could you rebase this PR on master and resolve the conflicts? Things have changed a bit, tests are now exposed in packages, but I'm not a fan of that because it pollutes the top-level namespace. legacyPackages would allow us to do .#tests.run.all, which I think is better. See discussions in https://github.com/nix-community/home-manager/pull/3972/files#r1189019541 and https://github.com/nix-community/home-manager/pull/3905.

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.

[ShamrockLee]

There are quite a few changes made in the master branch, and I need to figure out how to integrate with them.

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.