statically compile server binary

[shahidhk]

Server ships as a docker container since there are some dynamically linked components. If we can do a static compilation, server binary can be shipped as it is along with the container.

Helpful resource: https://www.fpcomplete.com/blog/2016/10/static-compilation-with-stack

[revskill10]

@shahidhk So we don't need docker for future deployment ?

[shahidhk]

@revskill10 It's just an idea :smile:

But yeah, if done there won't be a need for docker.

[ghost]

Hello,

Actually, I try to compile static binary on Ubuntu 18.04 LTS because Docker isn't a solution

server/Makefile, line 17 : stack build --fast --ghc-options '-j$(nproc) -optl-static -fPIC -optc-Os -split-objs' @shahidhk : we use the same link and I needed to do the swap about crtbeginT.o and crtbeginS.o

and I'm with this known error about libpq : https://github.com/lpsmith/postgresql-libpq/issues/17 If you have any idea

Regards

[shahidhk]

Hi @colundrum, this is a known issue and is a blocker for releasing a statically linked binary.

What you can do is install libpq-dev on the machine and then run the binary. It should work. You can take the binary out of the docker container or you can find it here: https://graphql-engine-cdn.hasura.io/server/latest/linux-amd64

[ghost]

@shahidhk Thanks a lot

On Fedora 30, now I'm at this step … I investigate about this error with ld

I think that Ubuntu's compilation of libpq doesn't permit but maybe Fedora can be a key : package "postgresql-static" exists in Fedora's repository

Unfortunately, hasura wants libpq-devel and don't use static library already packaged 😞 Maybe a solution will be to use https://github.com/postgres-haskell/postgres-wire

[nh2]

Hey, you may also consider using static-haskell-nix for this.

It has support for building stack projects, and would make it possible to build the static binary on any Linux distribution where the nix package manager can be installed.

It would also give the benefit that your static executable is fully reproducible because all dependencies, inclusive system dependencies and postgres .a files, are pinned (which they aren't if provided e.g. via Fedora's package manager).

[ghost]

Hello @nh2, Thanks for the link and I'll try it today

Edit : after an entire night on this, the linking wasn't successfully done due to libpq which doesn't find openssl … Other point is the cost of using nix … it recompiles absolutely all (kernel included), it's a time consuming which isn't compatible with quick rebuild and deploy In all case, libpq is still the problem … in or out of nix

[mthota15]

@shahidhk is there a server binary for mac os? is it possible to download specific version?

https://graphql-engine-cdn.hasura.io/server/latest/linux-amd64

[shahidhk]

@mthota15 Sorry, none that we publish. You can build from source by following the contributing guide. :smile:

Even https://graphql-engine-cdn.hasura.io/server/latest/linux-amd64 is not guaranteed to work on all Linux systems, especially when we add new system dependencies (like brotli for compression).

[nh2]

@colundrum

the linking wasn't successfully done due to libpq which doesn't find openssl …

That is a common situation, there is an easy fix by adding the library flags via pkgs-config, see https://github.com/nh2/static-haskell-nix/issues/57#issuecomment-544211259

Other point is the cost of using nix … it recompiles absolutely all (kernel included), it's a time consuming which isn't compatible with quick rebuild and deploy

It should build only once and then cache the results. It should not compile kernels (those should be fetched from cache.nixos.org). It will compile GHCs, but you can also use my binary cache as described on https://github.com/nh2/static-haskell-nix/#binary-caches-for-faster-building-optional and then it won't build those either.

[NilsParedes]

Is there any way to compile the server for windows?

[collinbachi]

Hi @shahidhk, can you elaborate on what the core issue is here?

I've been able to install the binary in several environments, as long as I also install libpq-dev. Is the issue just that the binary will require some (easy to install) outside dependencies?

I'm trying to figure out if I will be able to continue to install hasura on linux systems where Docker use is banned.

[tirumaraiselvan]

@collinbachi Yes, libpq is the only dependency for linux. You will also need updated CA certs if you want to interface with HTTPS domains for auth/remote-schemas/event-triggers etc.

cc: @0x777

[sycured]

Maybe I found a workaround for static binary about libpq: https://github.com/tweag/clodl

[revskill10]

I actually need static binary to locally develop with hasura, because i don't use docker on my machine now.

[natew]

Just want to add that developing on the M1 with Hasura is incredibly painful… queries run some 4-10x slower it seems, even using the forked M1 docker image. I tried building from source and run into issue after issue, I suppose I could build using a non-M1 and then share the binary but would need to borrow a computer to do so. If anyone has a Mac binary I’d happily pay just to have one.

Releasing Mac binaries just seems like a minimum, kind thing to do given the majority of developers use them and docker on Mac is basically fubar.

[nicoburns]

@tirumaraiselvan

Yes, libpq is the only dependency for linux

Do you know what the dependencies for macOS are? Even if there isn't a static binary available, a dynamically linked binary I could distribute along with instructions such as "run brew install libpq-dev" would be a lot better than the situation now (where we essentially have to recompile on every machine!).

I also wonder whether if it might be relatively easy to setup a homebrew formulae for the server? It seems like it should be possible to install any required dependencies as part of the homebrew formulae.

[natew]

Not sure if there are links in this but in the case it works, I was able to build following the mac instructions, here's the binary:

graphql-engine.zip

[gustavotrott]

Even graphql-engine-cdn.hasura.io/server/latest/linux-amd64 is not guaranteed to work on all Linux systems, especially when we add new system dependencies (like brotli for compression).

This binary works pretty well on my Ubuntu 20.04 server. The only problem for me is that I can't choose the version. Why not make this binary available in "Assets" of the release?

Unfortunately Docker is not an option for me because I'm already within a Docker, and docker-in-docker runs in the host pc, and I need Hasura to run in the same localhost of Pg database..

[manasag]

Hi Everyone, we cannot support HGE binary distribution. We recommend to build from source if this is required.