Add flake.lock & workflow to update it

[i077]

Description

This PR implements suggestions from #1913, adding a flake.lock file so we can nix run github:nix-community/home-manager without any hiccups. The nixpkgs input is currently set to track nixos-unstable. If you want to test this out, you can run nix run github:nix-community/home-manager/pull/1934/head, which should give you the help text.

To keep this file up to date, the PR also adds a GitHub workflow that

• Runs every Sunday and Wednesday at 03:51 (I just picked a random time)
• Runs nix flake update --commit-lock-file, committing as a nonexistent user called flakebot
• Pushes this commit ~~directly to master~~ to a new branch called flake-update with a custom token FLAKEBOT_TOKEN (this needs to be added to the repo)
• Opens a PR (or updates an existing PR) to merge this commit to master. This is done with the same custom token so that other workflows can be run against the PR.

Also updates the README to change the section on a flake-based config a bit.

I also wanted to modify the testing workflow to actually run home-manager's tests, but was having a bit of trouble exposing the tests under the checks output of the flake, so I will leave that out for now. ~~This means that the update workflow will just regularly push updates to master, whether they pass tests or not.~~

cc @berbiche since we were talking about this in the other PR.

Checklist

• [x] Change is backwards compatible.

• [x] Code formatted with ./format.

• [ ] Code tested through nix-shell --pure tests -A run.all.

• [ ] Test cases updated/added. See example.

• [ ] Commit messages are formatted like

  {component}: {description}
  
  {long description}
  

  See CONTRIBUTING for more information and recent commit messages for examples.

• If this PR adds a new module

  • [ ] Added myself as module maintainer. See example.

  • [ ] Added myself and the module files to .github/CODEOWNERS.

[teto]

at some point it was discussed to open a PR instead of an automatic merge. I preferred that scenario, is that doable through github actions ? nevermind the checks, they can be added later

[i077]

I can have the workflow checkout -B a new branch and open a new PR, that's what @kclejeune and I do on our nixos configs. My only concern was that I didn't want the repo to just have a bunch of flake update PRs, but if you're okay with that I can implement it.

edit: I should clarify: no more than 1 flake update PR will be open at a time, but it will require manual merging each time it gets opened (at most twice a week).

[teto]

edit: I should clarify: no more than 1 flake update PR will be open at a time, but it will require manual merging each time it gets opened (at most twice a week).

That seems the best option, would it close and open a new PR or force-push to the current one ?

[i077]

If the update action runs and there's already an existing PR, it will just force-push that branch. The -B switch in the checkout command basically "force creates" a new branch, even if it already existed in the remote, so the previous update commit just gets overwritten.

Also remember that a new secret containing a github token with push access to the repo needs to be made so we can trigger other workflows on the PR. I'd like to add the checks attribute to the flake with all of home-manager's tests so we can actually run them against the nixpkgs input in the update PR, but like I said I was having trouble with that so it can come later.

I'll update the workflow to push to a new branch later today.

[asymmetric]

This PR implements suggestions from #1913, adding a flake.lock file so we can nix run github:nix-community/home-manager without any hiccups.

If this is the only reason, wouldn't it make more sense to fix it on nix's side? Why does nix run try to save a lock-file in the first place? And shouldn't this be configurable somewhere?

[i077]

This PR implements suggestions from #1913, adding a flake.lock file so we can nix run github:nix-community/home-manager without any hiccups.

If this is the only reason, wouldn't it make more sense to fix it on nix's side? Why does nix run try to save a lock-file in the first place? And shouldn't this be configurable somewhere?

See the discussion in https://github.com/NixOS/nix/issues/4699.

[teto]

what is blocking ? looking forward to this to make lockstep updates in home-manager. Right now, when the nixos-unstable channel updates it can very well break home-manager. It would be better if we could choose what version of nixos-unstable works well via the flake.

[Mic92]

what is blocking ? looking forward to this to make lockstep updates in home-manager. Right now, when the nixos-unstable channel updates it can very well break home-manager. It would be better if we could choose what version of nixos-unstable works well via the flake.

From my understanding it requires a github token still? I can add one on every level.

[Mic92]

If you need any organisational changes please drop a message in #nix-community:bethselamin.de or #tmp-nix-community:numtide.com on matrix

[i077]

Has a token been added as a secret to this repo? What else needs to be done to move this PR forward?

[Mic92]

Has a token been added as a secret to this repo? What else needs to be done to move this PR forward?

No. I have received no token so far. I think the best solution would be if one of the home-manager maintainer create a bot account that we add to the nix-community organisation.

[stale[bot]]

Thank you for your contribution! I marked this pull request as stale due to inactivity. If this remains inactive for another 7 days, I will close this PR. Please read the relevant sections below before commenting.

If you are the original author of the PR

• GitHub sometimes doesn't notify people who commented / reviewed a PR previously when you (force) push commits. If you have addressed the reviews you can officially ask for a review from those who commented to you or anyone else.
• If it is unfinished but you plan to finish it, please mark it as a draft.
• If you don't expect to work on it any time soon, please consider closing it with a short comment encouraging someone else to pick up your work.
• To get things rolling again, rebase the PR against the target branch and address valid comments.

If you are not the original author of the PR

• If you want to pick up the work on this PR, please create a new PR and indicate that it supercedes and closes this PR.

[mainrs]

Wouldn't the correct behavior be to track the latest stable branch release (candidate) instead of nixos-unstable? And if a new release it cut, the channel is updated :thinking:

[kclejeune]

Wouldn't the correct behavior be to track the latest stable branch release (candidate) instead of nixos-unstable? And if a new release it cut, the channel is updated :thinking:

no, home manager's master branch tracks nixpkgs-unstable. There's a separate branch that tracks nixos-[stable]

[mainrs]

Wouldn't the correct behavior be to track the latest stable branch release (candidate) instead of nixos-unstable? And if a new release it cut, the channel is updated thinking

no, home manager's master branch tracks nixpkgs-unstable. There's a separate branch that tracks nixos-[stable]

Ah, thanks! I wasn't aware of that!

[teto]

@rycee could we move forward with this ?

[berbiche]

We need to decide what strategy to use for release branches and how many release branches will receive these updates.

We could drop the lock file in release branches but we would be required to use --no-write-lock-file when using HM. If we go for this solution, we can merge this PR as soon as the missing token is added.

If we ever needed to revert the changes or delete the lock file, the user's flake would automatically replace the nixpkgs used by Home Manager in its lockfile.

[Atry]

When I tried to execute:

nix run github:nix-community/home-manager --no-write-lock-file

I got:

warning: not writing modified lock file of flake 'github:nix-community/home-manager':
• Added input 'nixpkgs':
    'github:NixOS/nixpkgs/b283b64580d1872333a99af2b4cef91bb84580cf' (2022-05-01)
error: attribute 'defaultApp.x86_64-linux' should have type 'derivation'

However, if I execute:

nix shell github:nix-community/home-manager --no-write-lock-file -c home-manager

It works well.

I think we might also want to add a defaultApp to flake.nix

[terlar]

This is already provided (defaultApp). What you are seeing is a recent change in Nix 2.8 where the defaultApp needs to be a derivation as opposed to an attribute set that was the convention before.

[i077]

@DeterminateSystems have since released a GitHub Action that does what we would want, and it's probably better for home-manager to use that rather than maintain its own workflow. I can update this workflow to using that action instead.

It will use a default automated token for opening update PRs, but we'll still need to provide a personal auth token if we want to run tests against these PRs.

[ncfavier]

https://github.com/nix-community/home-manager/pull/2971 adds a flake.lock, so I guess it's blocked on this.

@DeterminateSystems have since released a GitHub Action that does what we would want, and it's probably better for home-manager to use that rather than maintain its own workflow. I can update this workflow to using that action instead.

Yes, I think that's better.

It will use a default automated token for opening update PRs, but we'll still need to provide a personal auth token if we want to run tests against these PRs.

This is currently not needed as the CI tests don't use the flake at all (maybe they should...).

This is already provided (defaultApp). What you are seeing is a recent change in Nix 2.8 where the defaultApp needs to be a derivation as opposed to an attribute set that was the convention before.

Fixed in https://github.com/nix-community/home-manager/pull/2971, see https://github.com/nix-community/home-manager/pull/2442#issuecomment-1133670487.

---

About the interval, I think twice a week is excessive considering that the updates require human intervention. Once every one or two weeks seems about right, maybe less often for release branches if we choose to update them (maybe release branches should follow the appropriate nixos- release branch instead of unstable?).

[i077]

This is currently not needed as the CI tests don't use the flake at all (maybe they should...).

My hope is that we'd later port rycee's home-manager testing framework to support flakes in a later PR. I was working on that very briefly, but stopped a while ago because of schoolwork.

About the interval, I think twice a week is excessive considering that the updates require human intervention. Once every one or two weeks seems about right, maybe less often for release branches if we choose to update them (maybe release branches should follow the appropriate nixos- release branch instead of unstable?).

I agree that release branches should follow the corresponding branch in nixpkgs. In particular, I think

• master should follow nixpkgs-unstable since home-manager is developed against that branch anyway, and
• release-xx.yy should follow nixos-xx.yy.

I suppose that means, going forward from release-21.11, we should change the nixpkgs input in flake.nix to follow the appropriate branch and cherry-pick backports on top of that as usual. (Cutting new release branches would involve updating flake.nix at that point as well.)

However, that would mean maintainers would have to review 2 PRs per run of this workflow (for master and current release branch).

[Mic92]

I added a secret for a github bot in this repo. If you use the GH_TOKEN_FOR_UPDATES secret than you will get a token of this bot https://github.com/home-manager-bot This bot can create branches on this repo

[Mic92]

The access of this bot is managed with terraform here: https://github.com/Mic92/dotfiles/blob/master/terraform/modules/github-push-bot/main.tf and here https://github.com/Mic92/dotfiles/blob/master/terraform/github-bots/main.tf

[ncfavier]

https://github.com/nix-community/home-manager/pull/2971 was merged, we have a flake.lock now.

[i077]

Thank you @ncfavier and @Mic92 for your help. A lot has happened since I opened this PR, so later today I will rebase my branch onto the tip of the master branch and redo most of the work so that it only adds the update workflow, since we now have a flake.lock.

As I mentioned somewhere earlier in this PR, future work will involve porting home-manager's tests to the checks output of flake.nix, so we can run CI tests against flake updates.

[i077]

Alright, branch rebased. Since most of the work in the rest of the old commits was done elsewhere, this PR now just implements that update workflow. PRs will be tagged with the "dependencies" label (like dependabot does) to distinguish it from other PRs.

[ShamrockLee]

The tests would be available through the legacyPackages output after ~#3024~ #3082 is merged. It can be run automatically by

$ nix develop .#tests.run.all

[teto]

tempted to merge this (and if it breaks stuff we'll deal with it). Any objection ?

[i077]

I'm fine with that. I will just draw attention to https://github.com/nix-community/home-manager/pull/1934#issuecomment-1134702268, in particular whether release branches should receive a similar update workflow, perhaps in a later PR:

I suppose that means, going forward from release-21.11, we should change the nixpkgs input in flake.nix to follow the appropriate branch and cherry-pick backports on top of that as usual. (Cutting new release branches would involve updating flake.nix at that point as well.)

However, that would mean maintainers would have to review 2 PRs per run of this workflow (for master and current release branch).

We don't have to address this now, but perhaps if we merge this PR we should consider doing the above on the next NixOS release.