Support binary prebuilt for main platforms?

[xaljer]

[rizsotto]

@xaljer good idea, but since it's a single developer project, I don't have the bandwidth to do it. Unless you want to help, I can't do it and would rely on distribution's packaging efforts.

[smhc]

Perhaps providing a Dockerfile or others to easily build via docker could be an option? I use the below for oracle linux 7.6.

FROM (oracle linux)
USER root
RUN yum-config-manager --add-repo http://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64 \
    yum-config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL7/SoftwareCollections/x86_64/ && \
    yum-config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/ \
    yum-config-manager --add-repo http://yum.oracle.com/repo/OracleLinux/OL7/developer_EPEL/x86_64
RUN yum install -y oraclelinux-release-el7 --nogpgcheck
RUN yum-config-manager --enable ol7_optional_latest
RUN yum -y install cmake3 --nogpgcheck
RUN yum -y install scl-utils devtoolset-9 --nogpgcheck
RUN yum -y install openssl-devel.x86_64 --nogpgcheck
RUN scl enable devtoolset-9 'git clone https://github.com/rizsotto/Bear.git --depth 1 && cd Bear && cmake3 -DENABLE_UNIT_TESTS=OFF -DENABLE_FUNC_TESTS=OFF . && make -j5 all || make install ||
make package || true'

although it may not work on 'vanilla' Oracle linux, and it also fails to build/install/package properly.. (I just roll my own package with the binaries I need).

[rizsotto]

@smhc This is interesting, I'm just wondering how do you use the docker image? Is it just building the package itself or using Bear from the image?

I would like to highlight that there are OS packages available for Bear.

[smhc]

I am simply extracting the executables out of the image/container after it has finished building. I doubt bear would work very well in a docker container on external processes due to the use of LD_PRELOAD etc. Note the build fails due to ctest errors etc - but it at least gives me the binaries.

Using a Dockerfile with docker allows reproducible builds on clean environments without needing to install dependencies locally. It's an ideal way of publishing build procedures that are reproducible by others. You'd probably want to use Ubuntu as the base.

It might be a good idea to have that OS package link in the README, building bear is now quite involved. Unfortunately I don't see any 3.0.0 packages that I can make use of.

[rizsotto]

I have copied the link from the README file. :) Yes, OS packaging are catching up with a delay of a few months.

I've provided an INSTALL file with build instructions and OS package dependency lists for a few distros. (I've found this is actually more than other software do to support users.) I am hesitant to put Docker files into the repo if I can't validate them in a CI.

To back to the original topic of this ticket. Do you know if GitHub action can use docker to pre-build packages and publish the artefacts? Could you help me to set that up?

[smhc]

Yes I think it's quite easy. I created a github docker workflow (just used the default): https://github.com/smhc/Bear/blob/master/.github/workflows/docker-image.yml

And a basic Dockerfile to build: https://github.com/smhc/Bear/blob/master/Dockerfile

I'm not sure where the artefacts are supposed to be pushed to and what artefacts you'd want. The image itself probably isn't that useful given it's unlikely bear would work containerised. But I suppose people could take the image and use both bear and gcc from it to compile their code to and generate a compile_commands.json.

You may also want to use a newer build based Dockerfile that removes all the compile time dependencies and just leaves you with the final binaries.

[rizsotto]

Woo, that's looks easy then. And I agree, we shall publish the binaries not the whole docker image. (Maybe can use the CMake package functionality.)

I think the workflow shall be executed on tags (I create releases from tags) and the release list has a place to hold the artefacts. That will probably needs some tool (github cli?) to upload.

[smhc]

The existing CI job is already building bear on ubuntu and could publish a release - it doesn't really need docker for that, does it? Docker does make it easier to build releases for other platforms and gives developers an easy way to build it themselves, so it's still ideal to have.

I made a small change to the Dockerfile to fix some interactivity issues but it likely needs a re-write. I'm not sure of the best practice of getting files out of a build. I've always just created a container and used 'cp', similar to : https://unix.stackexchange.com/questions/331645/extract-file-from-docker-image

Another option is to create an image with bear compiled, and then do the 'package install' as part of a container run with a mount such that it gets installed on a volume outside of the container.

It also needs an actual package creation step - I'm not aware of what this would be.

[smhc]

Fyi I added package creation and extraction, as well as storing the artefacts. It still needs work to build multiple platforms (different Dockerfiles) and the tie it into the release.

[rizsotto]

We have a few options here...

• Build static executables. Those are more portable between distributions. (Not dependent any other package of the system.) It requires a change on the CMake file, but probably can be controlled by an option/argument. It would require to build the dependent packages from source. (This is already solved problem.)
• Use CMake to create package. (cmake --build $build_dir --target package creates packages. CPACK_GENERATOR property can control what to build: .tar.gz, .deb or .rpm) This might need some tunes, because it was not really tested on any platform.
• Need to find where to install. (Not sure if Linux FHS is still a thing, but would like to be a good citizen.) I would pick a package name like BearHead or BearUpstream to avoid collision with existing packages and would install under /opt/Bear.

[smhc]

I think static executables are preferred if there are esoteric dependencies. At least for releases on the github release area.

I'm no packaging expert - but I do like the way clangd is released. You simply unzip it and it can find the dependent 'include' files relative to the clangd binary. This way it can be unzipped anywhere and work ok. I'm not sure how the various packagers put it together however.

It'd be nice if bear worked like this and didn't require a wrapper script to specify all the --interceptor etc options to find the right locations. Given that bear users have a high chance of being clangd users it makes sense to follow their model.

[xaljer]

appimage is also a good portable format to distribute software. neovim use appimage.

[smhc]

I've also seen many releases build against https://musl.libc.org/ for static binaries. It greatly increases the portability across different linux systems.

[edimetia3d]

I would vote for the "static link everything" solution, or build everything from source

For some reason, I have to use an ubuntu 16.04 as my dev machine,which doens't meet the dependency requriment. If "static link everything" is optional, I could build the binay in an ubuntu 20.04 docker, and copy the exec file out.

And, if possible, I thought "build everything from source" will be a much more portable solution.

[jacknicklenson]

+1 for static single binary executable releases for each platform like in:

• https://github.com/ninja-build/ninja/releases
• https://github.com/premake/premake-core/releases

[rizsotto]

@jacknicklenson happy to accept PR to make that happen.