Build icon indicating copy to clipboard operation
Build copied to clipboard

Use a docker container to make images build repeatable and deterministic

Open piffio opened this issue 7 years ago • 7 comments

Building images for Volumio requires a Debian system. This might not be available to the end user, or might have some unwanted versions of some packages. All this might render the build of images hard / impossible (like in my Fedora systems), or error prone and hard to reproduce.

This PR is a PoC to show how we could instead rely on a pre-built (and maintained by the Volumio team) docker image to perform the build.

This will have many advantages:

  1. Reduce friction / lower the barrier of entry for people who are willing to contribute and / or build their own custom images without having to have spare machines around

  2. Allow for easier integration with 3rd party tools for CI / Nightly builds and whatnot

  3. Make the builds deterministic since they will always happen on the "same system".

The dockerfile that builds the container is available in this repo, and there is a container published on Docker Hub that this PoC relies on to perform the build.

I am opening the PR mostly a way to start a conversation, since I find it easier to talk around some prototype implementation rather than in pure abstraction. The PR itself is actually just a one liner invocation of docker, to show how it could be used. If the idea is approved / considered of interest to the team, I am more than willing to turn the PoC in a proper (set of) PR with all the missing bits and pieces and either integrate with the build.sh script of provide some form of wrapper / alternative build command.

piffio avatar Feb 12 '18 20:02 piffio

Hi, sure this is an interesting thing, as the advantages are many. Really interested in seeing its developments

volumio avatar Feb 13 '18 12:02 volumio

Does it mean I should continue working on the concept?

If so, can you please share some thoughts on how should the docker build interact with the current build script?

Should we aim for replacing the current build or having the docker based build as a separate option?

piffio avatar Feb 13 '18 12:02 piffio

Yes, since it can be quite interesting, particularly if we can integrate it into a CI env for automated builds.

I would see it as a separate option, and it should basically use the same scripts as the current build (to allow portability of a single scripts set)

volumio avatar Feb 13 '18 12:02 volumio

Interesting. You may want to use latest qemu package (2.9.1 at least) to avoid some eventual arm build errors.

macmpi avatar Apr 07 '18 14:04 macmpi

I seems debian install will get qemu-user-static v2.8, which is not sufficient: in my tests on std Volumio build, 2.8 did not fix all issues.

macmpi avatar Apr 07 '18 18:04 macmpi

Hey @macmpi , thanks for the comments. I have fixed the issue with qemu in the build, and everything seems to work nicely with the exception of the build for bbb (beagleboard) which I need to investigate more in depth

piffio avatar Apr 08 '18 21:04 piffio

FWIW -- volumioOS and friends work quite reliably in an containerised environment. Moving to CI builds might be something to seriously consider in the future.

ashthespy avatar Oct 06 '21 08:10 ashthespy