Reduce the number of derivations

[hamishmack]

trafficstars

• Share the default setup executable.
• Make configFiles and ghc outputs of the component derivation.
• Use propagatedBuildInputs and pkgsHostTarget for component dependencies.

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• ci/hydra:Cardano:haskell-nix:required

[hamishmack]

bors try

[hamishmack]

bors try-

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• ci/hydra:Cardano:haskell-nix:required

[michaelpj]

Use propagatedBuildInputs and pkgsHostTarget for component dependencies.

Is this a good idea? This means that if you pull in our wrapped GHC for any other purposes you'll get all the components pulled in too, right? And does it actually save us derivations? Presumably you still have the same number of derivations, they're just tracked differently?

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix

[hamishmack]

Is this a good idea?

The gain I was hoping to get here was reducing the amount of strings concatenation and the overall size of the strings in the component derivations. Currently we currently concat the output paths of the dependencies and put them in configurePhase string. Ultimately I would like to use the same static configure.sh and build.sh for all components, this is a first step towards that.

This means that if you pull in our wrapped GHC for any other purposes you'll get all the components pulled in too, right?

Good point. Although the GHC derivation did not change, executable components do include the propagatedBuildInputs. I'll change it to use regular buildInputs for executable components. I think only the library components need propagatedBuildInputs.

And does it actually save us derivations?

No

Presumably you still have the same number of derivations, they're just tracked differently?

Yes. I am hoping the more standard nix solution may be better optimised.

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix

[michaelpj]

Maybe worth doing 1+2 separately since they're maybe less controversial and also less invasive?

[hamishmack]

2+3 are fairly closely linked. It might be hard to split and merge them. We could probably do 1 on its own (there is a branch with that), but all of these changes require rebuilding all the GHCs on hydra.iohk.io and it would be nice if we only had to do that once.

[hamishmack]

bors try

[iohk-bors[bot]]

try

Timed out.

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix

[L-as]

Why not just make each derivation more efficient? If it's several orders of magnitude of difference, maybe, but otherwise, making the derivations more light weight and possibly optimising the daemon seems more efficient to me.

[hamishmack]

Why not just make each derivation more efficient?

How?

[L-as]

Avoiding stdenv is probably a big one.

[hamishmack]

Avoiding stdenv is probably a big one.

Not sure I understand. Can you give and example of something in the haskell.nix code we should be avoiding?

The goal here is to reduce the eval time when no code needs to built. My aim was to reduce the amount and size of the derivations in the hope that eval would be faster over all.

Once this change is merged I was going to look into how much could be saved by move the large configurePhase and buildPhase and installPhase strings into static shell scripts that use env variables as inputs (that is why this change uses pkgsHostTarget environment variable).

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• ci/hydra:Cardano:haskell-nix:required

[hamishmack]

bors try

[iohk-bors[bot]]

try

Build failed:

• buildkite/haskell-dot-nix