haskell.nix
haskell.nix copied to clipboard
aarch64-darwin: Passing `system = "x86_64-darwin"` still triggers build of aarch64 GHC
We tried to use haskell.nix
on an M1 Mac with Rosetta 2 installed, by overriding the system
parameter:
let
# NB: We are _not_ using flakes here
pkgs = (import sources."haskell.nix" { system = "x86_64-darwin"; }).pkgs;
project = pkgs.haskell-nix.cabalProject { ... };
in
project.shellFor { ... }
This still triggers a build of GHC for aarch64-darwin
, which even given M1 performance takes a very long time. I suspect that this is due to the eval-packages.nix
overlay forcing the use of builtins.currentSystem
, when building various tools for the IFDs:
https://github.com/input-output-hk/haskell.nix/blob/f44f0845b29bf9caf7884510f4b17e9423673fc9/overlays/eval-packages.nix#L9-L13
I'm not clear on the best way to fix this - we certainly don't want to plumb system
through to this import, as that would break cross-compilation. We could:
- Add another parameter which can override the
system
detection ineval-packages.nix
? - Special-case
system = "x86_64-darwin"
whenbuiltins.currentSystem == "aarch64-darwin"
? - Something else?
I'm happy to try and PR this, but I'll need some guidance.
CC @Tristano8
Ohh, that's a nasty bug indeed. We've probably not hit this before because there were only very few systems that would support multiple archs at the same time.
What happens if we drop the builtins.currentSystem
outright and go always with final.buildPackages.system
?
The whole reason for the eval-packages
dance is so that when you are cross-compiling from X->Y, you can still evaluate on X! Otherwise you simply can't e.g. evaluate a darwin cross derivation on a linux machine, which really sucks. The typical usecase is "I don't have a darwin builder (hydra will do that when I push) but I want to check that everything evaluates properly locally". (In fact, I think this was added because I complained about it :) )
In many ways it would be simpler and more logical to do the IFDs on the build platform, but it does make developing miserable.
We've found a workaround for ARM Macs: set system = x86_64-darwin
in nix.conf
, rather than passing it as an argument (and optionally, extra-platforms = aarch64-darwin
). This has the sad side-effect of forcing Rosetta 2 for everything else you get from Nix, but gets you IFD evals without building GHC, and without forcing rebuilds of everything else.
I agree that eval-packages
exists for a good reason, and for >=99% of non-flake uses, builtins.currentSystem
is the right thing to use. I think this problem goes away when one of two things happen:
- IOHK Hydra gets an
aarch64-darwin
builder (so you can evaluate IFD without building AArch64 GHC, but probably removes a lot of the reason to passsystem = "x86_64-darwin"
tohaskell.nix
?) - We add a parameter somewhere which can override the
system
used byevalPackages
.
@JackKelly-Bellroy you can try
https://cache-1.zw3rk.com/
loony-tools:pr9m4BkM/5/eSTZlkQyRt57Jz7OMBxNSUiMC4FkcNfk=
as a binary cache. (At some point I should generate a new key called zw3rk or something.
https://ci.zw3rk.com does have 3 m1 minis and does build haskell.nix as well (as well as a ton of other stuff).
@angerman I tried this cache myself and it still tries to build GHC on my M1 Mac
Hi, are m1 macs still unsupported by the binary cache?
I've added the below to my darwin-configuration.nix
file:
# Binary Cache for Haskell.nix
nix.binaryCachePublicKeys = [
"hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ="
"loony-tools:pr9m4BkM/5/eSTZlkQyRt57Jz7OMBxNSUiMC4FkcNfk="
];
nix.binaryCaches = [
"https://cache.iog.io"
"https://cache.zw3rk.com"
];
I am trying to use ghc901
in a simple hello-world project (flake below):
{
inputs.haskellNix.url = "github:input-output-hk/haskell.nix";
inputs.nixpkgs.follows = "haskellNix/nixpkgs-unstable";
inputs.flake-utils.url = "github:numtide/flake-utils";
outputs = { self, nixpkgs, flake-utils, haskellNix }:
flake-utils.lib.eachSystem [ "aarch64-darwin" ] (system:
let
overlays = [ haskellNix.overlay
(final: prev: {
# This overlay adds our project to pkgs
helloProject =
final.haskell-nix.project' {
src = ./.;
compiler-nix-name = "ghc901";
# This is used by `nix develop .` to open a shell for use with
# `cabal`, `hlint` and `haskell-language-server`
shell.tools = {
cabal = {};
hlint = {};
haskell-language-server = {};
};
# Non-Haskell shell tools go here
shell.buildInputs = with pkgs; [
nixpkgs-fmt
];
# This adds `js-unknown-ghcjs-cabal` to the shell.
# shell.crossPlatforms = p: [p.ghcjs];
shell.withHoogle = true;
};
})
];
pkgs = import nixpkgs { inherit system overlays; inherit (haskellNix) config; };
flake = pkgs.helloProject.flake {
# This adds support for `nix build .#js-unknown-ghcjs-cabal:hello:exe:hello`
# crossPlatforms = p: [p.ghcjs];
};
in flake // {
# Built by `nix build .`
defaultPackage = flake.packages."hello:exe:hello";
});
}
![Screenshot 2022-09-14 at 10 32 01](https://user-images.githubusercontent.com/8767122/190117946-7134b195-c68b-4d1f-ae8f-d6968cf00eb0.png)
I can see that it is querying the cache at https://cache.zw3rk.com
, yet it still seems to try to build GHC:
![Screenshot 2022-09-14 at 10 35 21](https://user-images.githubusercontent.com/8767122/190118748-2a52bdd7-109c-44a4-9078-e73e4fc2b72f.png)
Frustratingly, some time later I get a failure:
[0/1 built, 0.0 MiB DL] querying easy-file-lib-easy-file-0.2.2-haddock-doc on https://cache.zw3rk.com
error: builder for '/nix/store/78p7gbqi2lswl144r5zjmjzjgplnkxk4-ghc-lib-parser-lib-ghc-lib-parser-9.2.3.20220527.drv' failed with exit code 1;
last 10 log lines:
> 'vortex' is not a recognized processor for this target (ignoring processor)
I am asking myself the same question as @G3zz before me, is aarch64-darwin expected to be found in the binary caches at this time? I can't seem to get it to work no matter what I do.
I tried it not using flakes and I got a cache hit - something in the flake causes a cache miss
@G3zz Are you 100% sure you’re using the same configuration and version of Haskell.nix? Could you please provide some example code so that I can see if I can reproduce it?
I don't think we build ghc901
for all systems (even on ci.zw3rk.com). Have you tried switching to ghc902
?
nix-diff
is really good in these situations. If you can get the .drv
for pkgs.compiler.ghc902
on the cached and non cached versions then nix-diff
should hopefully reveal what is different.
I'm not clear on the best way to fix this - we certainly don't want to plumb
system
through to this import, as that would break cross-compilation. We could:
- Add another parameter which can override the
system
detection ineval-packages.nix
?- Special-case
system = "x86_64-darwin"
whenbuiltins.currentSystem == "aarch64-darwin"
?- Something else?
We removed overlays/eval-packages.nix
and instead evalPackages
is now an optional argument you can pass to the project. You can also pass just evalSystem = "x86_64-darwin"
to the project (and the default evalPackages
will then use that).
Because builtins.currentSystem
requires --impure
the default evalSystem
is pkgs.buildPackages.system
.
Flakes must make a choice of evalSystem
(or system for evalPackages
) and the options are:
- Pass nothing and it will match
buildPackages.system
(if the flake has multiple systems you will need builders for them all to runnix flake show
). - Hard code it to some system you know you will have a builder for.
- Pass in
builtins.currentSystem
. - Some logic like
evalSystem = if system == "aarch64-darwin" then "x86_64-darwin" else system;
orevalSystem = if builtins.currentSystem == "aarch64-darwin" then "x86_64-darwin" else system;
.
If you choose to use builtins.currentSystem
keep in mind:
- You will have to pass
--impure
to nix. - This will prevent the nix from caching the results of evaluations.
- You might want to check how long things like
nix develop
take to launch.
If you do use builtins.currentSystem
then you can pass --system x86_64-darwin
on the command line or add system = x86_64-darwin
to ~/.config/nix/nix.conf
and that will cause make it so everything runs as if you had a x86 only system with an aarch64 builder.
% nix repl --system x86_64-darwin
Welcome to Nix 2.12.0pre20220921_f704c27. Type :? for help.
nix-repl> builtins.currentSystem
"x86_64-darwin"
nix-repl> :q
% nix repl --system aarch64-darwin
Welcome to Nix 2.12.0pre20220921_f704c27. Type :? for help.
nix-repl> builtins.currentSystem
"aarch64-darwin"
nix-repl> :q