neon icon indicating copy to clipboard operation
neon copied to clipboard

Separate Postgres build dir from installation dir.

Open hlinnaka opened this issue 3 years ago • 2 comments

Previously, Postgres was built in 'tmp_install/build', and installed into 'tmp_install'. In the CI, the 'build' directory was included in the final neon.tgz artifact that includes all the necessary binaries. That was unnecessary, the intermediate build results are not needed, only the final binaries. Separate the build directory so that the Postgres build happens in 'build', and it is installed into 'tmp_install'. That makes the final neon.tar.zst artifact smaller.

hlinnaka avatar Jul 20 '22 07:07 hlinnaka

I made this change in the Makefile, so that you get:

build/
tmp_install/bin
tmp_install/lib
...

Instead of:

tmp_install/build
tmp_install/bin
tmp_install/lib
...

The 'build' directory doesn't need to be included in the final artifact, it only contains build-time object files etc.

An alternative would have been to modify the github workflow yaml file, to exclude tmp_install/build from the tarball. But I think it makes sense to separate the final installation directory from the build directory anyway. I also considered moving the final PG install dir under target, e.g. target/pg_install, since that's where the Rust binaries go. But I wasn't sure if that would confuse some other tools. Maybe we should have a new dist directory, where we copy all the final binaries. Or a make install target that copies all the final binaries to a user-specified directory. But I think this will do for now.

hlinnaka avatar Jul 20 '22 07:07 hlinnaka

I like the idea in general, yet seems that some test fails due to misconfiguration, that needs fixing.

If we plan to add more extensions to postgres, would we need to add more cryptic paths into the tests? Does not seem to scale well, but I might be overestimating our existing infra for this task slightly_smiling_face

Yeah, I've done ~ 10 iterations of this, fixing failure, force-pushing, waiting for the CI to fail again. And it's still failing. I've got a feeling that we need a bigger refactoring of these scripts to make it all less brittle.

hlinnaka avatar Jul 20 '22 12:07 hlinnaka

As discussed moving to draft.

LizardWizzard avatar Mar 21 '23 15:03 LizardWizzard