raspberry-pi-nix icon indicating copy to clipboard operation
raspberry-pi-nix copied to clipboard

Cachix not working - recompiling the kernel

Open MLFlexer opened this issue 1 year ago • 3 comments

I am trying to nixos-rebuild buildmy raspberry pi 5, but whenever I update my flake inputs I have to recompile the kernel. It says in the readme that it is possible to use the cachix cache, but it does not seem to work for me or I'm not doing it correctly... I have the following set in my nixos config:

  nix = {
    package = pkgs.nixVersions.stable;
    extraOptions = "experimental-features = nix-command flakes";
    settings = {
      trusted-users = [ "root" "mlflexer" ];
      trusted-substituters = [ "https://nix-community.cachix.org" ];
      trusted-public-keys = [
        "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
      ];
    };
  };

And I don't get any warning about it not using the cache, but I can see it is recompiling the kernel and that it takes forever to do. Here is the output when running:

nixos-rebuild build --flake .#rpi5 -v --show-trace
building the system configuration...
Building in flake mode.
$ nix --extra-experimental-features nix-command flakes build .#nixosConfigurations."rpi5".config.system.build.toplevel -v --show-trace
warning: Git tree '/home/mlflexer/repos/.dotfiles' is dirty
Using saved setting for 'substituters = https://cache.nixos.org https://nix-community.cachix.org' from ~/.local/share/nix/trusted-settings.json.
Using saved setting for 'trusted-public-keys = nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=' from ~/.local/share/nix/trusted-settings.json.
Using saved setting for 'trusted-users = root mlflexer' from ~/.local/share/nix/trusted-settings.json.

Am I not adding it correctly or is it just not available in the cache?

Also thanks for this awesome work! Let me know if you need more info 😄

MLFlexer avatar Feb 20 '25 18:02 MLFlexer

Hi, I have a similar setup to you, but I'm running a raspberry pi 4B. (edit: I'm also using the cachix cache)

I ran into a similar problem as you with the SD image taking forever to build. I saw that it was stuck on libcamera, so this is where I focused. I copied the configuration in ./example/default.nix, but I ended up disabling the libcamera overlay (see below).

This made the build actually buildable, and disabling this didn't matter to me anyway because I don't use cameras with my pi.

{
  ...
  raspberry-pi-nix = {
    board = "bcm2711";
    kernel-version = "v6_6_51";
    libcamera-overlay = {
      enable = false; # set to false (enabled by default)
    };
  };
  ...
}

Hope this helps (though I have the same concern about build times!!).

suasuasuasuasua avatar Feb 21 '25 04:02 suasuasuasuasua

Thanks for the suggestion! Unfortunately it still takes a long time for me to compile my own flake, even without the libcamera-overlay. I have however found a way to speed it up to only taking ~2 hours using GitHub actions, by adding the following GitHub actions workflow script:

name: "rpi5_build"
on:
  workflow_dispatch:
jobs:
  tests:
    runs-on: ubuntu-24.04-arm
    steps:
    - uses: actions/checkout@v4
    - uses: cachix/install-nix-action@v25
      with:
        nix_path: nixpkgs=channel:nixos-unstable
    - uses: cachix/cachix-action@v14
      with:
        name: mlflexer
        authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
    - name: Build the rpi5 flake
      run: nix build .#nixosConfigurations.rpi5.config.system.build.toplevel

Here you need to create a cache on Cachix and then set your secrets key. Then you need to add your caches as a substituters and the trusted-public-keys of the cache.

MLFlexer avatar Feb 22 '25 07:02 MLFlexer

See this comment https://github.com/nix-community/raspberry-pi-nix/issues/95#issuecomment-2577956043

tstat avatar Feb 24 '25 15:02 tstat