selfhostblocks icon indicating copy to clipboard operation
selfhostblocks copied to clipboard

SelfHostBlocks cannot be deployed from macOS (aarch64-darwin) to Linux servers (x86_64-linux) using deploy-rs due to flake evaluation requiring x86_64-linux during the local evaluation phase.

Open nikhilmaddirala opened this issue 5 months ago • 3 comments

Summary

SelfHostBlocks cannot be deployed from macOS (aarch64-darwin) to Linux servers (x86_64-linux) using deploy-rs due to flake evaluation requiring x86_64-linux during the local evaluation phase.

Environment

  • Host System: macOS (aarch64-darwin) - Apple Silicon Mac
  • Target System: Linux server (x86_64-linux)
  • Deployment Tool: deploy-rs with remoteBuild = true
  • SelfHostBlocks Version: Latest (commit c9ee41b1d413b460bd652845484c1b1338b8ce5c)

Expected Behavior

Should be able to deploy SelfHostBlocks from macOS to Linux server using deploy-rs with remote building, similar to how regular nixpkgs configurations work.

Actual Behavior

Deployment fails during flake evaluation phase with cross-compilation error:

error: a 'x86_64-linux' with features {} is required to build '/nix/store/cpyipkyc9hy21d7i59k7xsy1f93l00aq-nixpkgs-patched.drv', but I am a 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}

Reproduction Steps

  1. Create flake on macOS using selfhostblocks:
{
  inputs = {
    selfhostblocks.url = "github:ibizaman/selfhostblocks";
    # ... other inputs
  };
  
  outputs = inputs: {
    nixosConfigurations.myserver = inputs.selfhostblocks.lib.x86_64-linux.patchedNixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        inputs.selfhostblocks.nixosModules.x86_64-linux.default
        # ... configuration modules
      ];
    };
    
    deploy.nodes.myserver = {
      hostname = "example.com";
      remoteBuild = true;  # This doesn't help with evaluation
      # ... deploy config
    };
  };
}
  1. Attempt deployment:
nix run github:serokell/deploy-rs .#myserver
  1. Error occurs during evaluation phase before any remote operations

Root Cause Analysis (warning: AI generated)

The issue occurs because:

  1. SelfHostBlocks uses patched nixpkgs that requires x86_64-linux evaluation
  2. deploy-rs evaluates flakes locally before sending to remote system
  3. Patched nixpkgs evaluation fails on macOS during this local evaluation phase
  4. Remote building doesn't help because evaluation happens before building

nikhilmaddirala avatar Aug 04 '25 03:08 nikhilmaddirala

Seems similar to: https://github.com/ibizaman/skarabox/issues/90

nikhilmaddirala avatar Aug 04 '25 03:08 nikhilmaddirala

FWIW with the following config and using nix-darwin, I can build it:

      nix.distributedBuilds = true;
      nix.linux-builder = {
        enable = true;
        ephemeral = true;
        maxJobs = 4;
        supportedFeatures = [ "kvm" "benchmark" "big-parallel" "nixos-test" ];
        systems = [
          # "x86_64-linux"
          "aarch64-linux"
        ];
        config = {
          virtualisation = {
            darwin-builder = {
              diskSize = 40 * 1024;
              memorySize = 8 * 1024;
            };
            cores = 6;
          };
        };
      };

      nix-rosetta-builder = {
        enable = true;
        # onDemand = true;
      };

ibizaman avatar Aug 19 '25 22:08 ibizaman

FWIW with the following config and using nix-darwin, I can build it:

      nix.distributedBuilds = true;
      nix.linux-builder = {
        enable = true;
        ephemeral = true;
        maxJobs = 4;
        supportedFeatures = [ "kvm" "benchmark" "big-parallel" "nixos-test" ];
        systems = [
          # "x86_64-linux"
          "aarch64-linux"
        ];
        config = {
          virtualisation = {
            darwin-builder = {
              diskSize = 40 * 1024;
              memorySize = 8 * 1024;
            };
            cores = 6;
          };
        };
      };

      nix-rosetta-builder = {
        enable = true;
        # onDemand = true;
      };

Thanks! Could you try to repro with remote builder via deploy-rs instead of Rosetta builder?

Unfortunately I can't repro what you did due to unrelated issues preventing me from using linux-builder on my darwin machine.

nikhilmaddirala avatar Aug 21 '25 04:08 nikhilmaddirala