wire-server
wire-server copied to clipboard
Generate/write nix derivations for all the haskell code
Most of these nix files are generated using ./hack/bin/generate-nix-overrides.sh
and ./hack/bin/generate-local-nix-overrides.sh
.
The generate-nix-overrides.sh
script reads pins.yaml
(Yes, I invented another YAML thing :cry:) and generates nix derivations using cabal2nix
and puts them in nix/haskell-overrides/
. It also generates nix/haskell-overrides.nix
which contains the overrides required to make these derivation part of any pkgs.haskell.packages.<ghc>
package set.
The generate-local-nix-overrides.sh
script finds all the cabal files in the repository and generates a nix derivation for each package using cabal2nix
and puts each derivation next to the package itself in a default.nix
file. It also generates nix/local-overrides.nix
similar to the haskell-overrides.nix
.
The build.nix
at the root of the repo, combines these two overrides and also makes sure the packages which are not pinned by pins.yaml
work. The reason for not pinning these packages is that we don't actually care if these are new and we should keep upgrading them until we can. This way we can keep the pinned things to a minimum.
TODO:
- [ ] Add details about how shellFor and docker images work.
- [ ] Make building without optimization configurable
- [ ] Ensure that HLS works
The build.nix at the root of the repo
This seems to be missing from the PR
The build.nix at the root of the repo
This seems to be missing from the PR
Added it now.
How much of this is generated vs. written? I naturally recoil at treating anything generated as source code and checking it in, though I certainly understand it's the path of least resistence in some cases.
How much of this is generated vs. written? I naturally recoil at treating anything generated as source code and checking it in, though I certainly understand it's the path of least resistence in some cases.
All the new files in the PR except pins.yaml
, build.nix
and the 3 shell scripts in hack/
are generated. For all the default.nix
files, we could use callCabal2nix
function, but a file with list of those calls to the function for each package would either need to be generated or manually written. For all the pinned things, I think it is possible to write something with callCabal2nix
and fetchGit
, but it will be a little effort to do this. That said, I think this will also make things harder to debug, so I would still prefer just committing these files and have CI ensure that these files are not out of date.
A few questions, sorry if they are dumb, but I still struggle with nix things.
- Why are you generating all these nix files? Can't cabal2nix or whatever just read the cabal file dynamically and create the corresponding derivation?
- Are we dropping stackage now? Is this necessary? Couldn't we be using the same pins as before and avoid the trouble of finding a consistent choice of versions? This might become an issue in the future.
- Relatedly, why do we need the hackage portion of
pins.yaml
? Can't we get versions from the existing yaml files (i.e.cabal.project.freeze
).
Why are you generating all these nix files? Can't cabal2nix or whatever just read the cabal file dynamically and create the corresponding derivation?
There exists a callCabal2nix
but it only works for local paths. I think we can write something similar to it to avoid generating the files. We will still have to generate a file which calls this new function with the right parameters and creates the overrides.
Are we dropping stackage now? Is this necessary? Couldn't we be using the same pins as before and avoid the trouble of finding a consistent choice of versions? This might become an issue in the future.
The pkgs.haskell.packages.ghc8107
already follows some stackage pin, but it refers to a nightly snapshot. The overrides are similar to our overrides written in the stack.yaml.
Relatedly, why do we need the hackage portion of
pins.yaml
? Can't we get versions from the existing yaml files (i.e.cabal.project.freeze
).
We want to not rely on stack.yaml
so I chose to not read it. The cabal.project.freeze
file is not a YAML file and I didn't feel like writing a parser and it contains more versions than we want to pin as we want to follow some stackage snapshot. So, for this approach to work we have to delete the cabal.project.freeze
file along with the source-repository-package
sections in the cabal.project
file.
We could also try to get rid of the pins.yaml
file if we can get a function similar to callCabal2nix
. If we do that we can write all our pins in the build.nix
file.
I rebased on latest develop
and updated pins.yaml
to track cabal.project in the rebased commit.
Currently build failures:
Configuring partial-isomorphisms-0.2.3.0...
CallStack (from HasCallStack):
$, called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:1024:20 in Cabal-3.2.1.0:Distribution.Simple.Configure
configureFinalizedPackage, called at libraries/Cabal/Cabal/Distribution/Simple/Configure.hs:477:12 in Cabal-3.2.1.0:Distribution.Simple.Configure
configure, called at libraries/Cabal/Cabal/Distribution/Simple.hs:625:20 in Cabal-3.2.1.0:Distribution.Simple
confHook, called at libraries/Cabal/Cabal/Distribution/Simple/UserHooks.hs:65:5 in Cabal-3.2.1.0:Distribution.Simple.UserHooks
configureAction, called at libraries/Cabal/Cabal/Distribution/Simple.hs:180:19 in Cabal-3.2.1.0:Distribution.Simple
defaultMainHelper, called at libraries/Cabal/Cabal/Distribution/Simple.hs:116:27 in Cabal-3.2.1.0:Distribution.Simple
defaultMain, called at Setup.hs:2:8 in main:Main
Setup: Encountered missing or private dependencies:
template-haskell >=2.17
i checkout this branch out on a fresh clone. direnv allow
succeeded after 40 minutes. make c
had this to say:
Configuring wire-message-proto-lens-0.1.0...
[ 9 of 16] Compiling Test.Schema.Util ( test/Test/Schema/Util.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/Util.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/Util.dyn_o )
[10 of 16] Compiling Test.Schema.UserSpec ( test/Test/Schema/UserSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/UserSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/UserSpec.dyn_o )
[11 of 16] Compiling Test.Schema.ResourceSpec ( test/Test/Schema/ResourceSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/ResourceSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/ResourceSpec.dyn_o )
[12 of 16] Compiling Test.Schema.PatchOpSpec ( test/Test/Schema/PatchOpSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/PatchOpSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/PatchOpSpec.dyn_o )
setup: filepath wildcard 'generic-message-proto/proto/otr.proto' does not
match any files.
[13 of 16] Compiling Test.Schema.MetaSchemaSpec ( test/Test/Schema/MetaSchemaSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/MetaSchemaSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/MetaSchemaSpec.dyn_o )
[14 of 16] Compiling Test.Schema.GroupSpec ( test/Test/Schema/GroupSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/GroupSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/GroupSpec.dyn_o )
[15 of 16] Compiling Test.Schema.AuthenticationSchemeSpec ( test/Test/Schema/AuthenticationSchemeSpec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/AuthenticationSchemeSpec.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Test/Schema/AuthenticationSchemeSpec.dyn_o )
[16 of 16] Compiling Main ( test/Spec.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Main.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec-tmp/Main.dyn_o )
Linking /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/hscim-0.3.6/t/spec/build/spec/spec ...
[26 of 26] Compiling Wire.Swagger ( src/Wire/Swagger.hs, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/types-common-0.16.0/build/Wire/Swagger.o, /home/mf/tmp/wire-server/dist-newstyle/build/x86_64-linux/ghc-8.10.7/types-common-0.16.0/build/Wire/Swagger.dyn_o )
cabal: Failed to build wire-message-proto-lens-0.1.0 (which is required by
test:wire-api-tests from wire-api-0.1.0, test:wire-api-golden-tests from
wire-api-0.1.0 and others).
@fisx You need to fetch submodules in wire-server too, git submodule update --init --recursive
there was a disk space issue. with enough disk space, make full-clean c ci
still fails in the same place.
git submodule update --init --recursive
that sounds helpful! is there a way to make this automatic? if not i'll add this trick to the PSA we already give i make c
fails.
there was a disk space issue. with enough disk space,
make full-clean c ci
still fails in the same place.
What is the eror message? It should be different when you've pulled the submodules. For pulling submodules automatically I think https://stackoverflow.com/a/49427199
there was a disk space issue. with enough disk space,
make full-clean c ci
still fails in the same place.What is the eror message? It should be different when you've pulled the submodules. For pulling submodules automatically I think https://stackoverflow.com/a/49427199
trying with submodules now, hopeful this will work.
git submodule update --init --recursive
that sounds helpful! is there a way to make this automatic? if not i'll add this trick to the PSA we already give i
make c
fails.
https://github.com/wireapp/wire-server/pull/2761/commits/c9acc5aa574c017973f130734c6334ba92a27300
direnv allow succeeded after 40 minutes
I'm wondering why it took so long. We don't build the devShell on ci I think, however all the wireServerPackages
should be in the cache, right @akshaymankar ?
direnv allow succeeded after 40 minutes
I'm wondering why it took so long. We don't build the devShell on ci I think, however all the
wireServerPackages
should be in the cache, right @akshaymankar ?
I'm not sure and I lost the terminal and no logs, but I don't remember any building. Maybe the nix-cache server is just slow? Haven't done the numbers to decide how plausible that is. I'm on a good uplink, though.
another weird thing: every shell command output is not enriched with this direnv noise:
~/tmp/wire-server$ ls
cabal.project changelog.d charts dist-newstyle hack libs Makefile package-defaults.yaml SECURITY.md shell.nix tools
cassandra-schema.cql CHANGELOG.md deploy docs hie.yaml LICENSE nix README.md services snapshots
direnv: loading ~/tmp/wire-server/.envrc
direnv: loading ~/tmp/wire-server/.env/.env/dev-shell.sh
direnv: loading ~/tmp/wire-server/.env/profile-env/.env/profile-env/.profile
./.envrc:40: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +DETERMINISTIC_BUILD +HOST_PATH +IN_NIX_SHELL +LC_ALL +LD +LOCALE_ARCHIVE +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_BUILD_TOP +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_GHC +NIX_GHCPKG +NIX_GHC_DOCDIR +NIX_GHC_LIBDIR +NIX_HARDENING_ENABLE +NIX_INDENT_MAKE +NIX_LDFLAGS +NIX_PATH +NIX_PKG_CONFIG_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_STORE +NM +OBJCOPY +OBJDUMP +PKG_CONFIG +PKG_CONFIG_PATH +PYTHONHASHSEED +PYTHONNOUSERSITE +PYTHONPATH +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +XDG_DATA_DIRS +_PYTHON_HOST_PLATFORM +_PYTHON_SYSCONFIGDATA_NAME +buildInputs +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +installPhase +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~WINDOWID
can this be turned off?
The build process was killed (i.e. SIGKILL). The
typical reason for this is that there is not enough memory available (e.g. the
OS killed a process using lots of memory).
trying again, with less system noise.
sigh
I'm wondering why it took so long. We don't build the devShell on ci I think, however all the
wireServerPackages
should be in the cache, right @akshaymankar ?
@fisx do you have https://wire-server.cachix.org/ in your nix substituters?
another weird thing: every shell command output is not enriched with this direnv noise:
This happened to me once, I deleted .direnv
in wire-server and it went away. I will look deeper into direnv to see if this step is unnecessary.
The build process was killed (i.e. SIGKILL). The typical reason for this is that there is not enough memory available (e.g. the OS killed a process using lots of memory).
trying again, with less system noise.
sigh
forgot to copy cabal.profile.local into the fresh clone. i'm not going to miss a single trap... :)
forgot to copy cabal.profile.local into the fresh clone. i'm not going to miss a single trap... :)
Oh yeah, compiling with -O2
requires a lot of memory right now :(