Eliminate Dockerfile.in and rootfs.yml.in

[deitch]

We have a few places where we do *.in files, which are then parsed somehow, primarily some commands called from the Makefile and often involving parse-pkgs.sh, to generate the actual files. This is intended as a discussion issue where we can discuss ways to simplify the flow, and if those are any better.

The intent is to get to a "preferred flow", i.e. what we would like this to look like, and then we can look at implementation. It is not at all clear that any of these is better than the current one, but should raise the issues.

We have two categories of .in files: Dockerfile.in and linuxkit yml.in. Both are there for the same reason. Some of our packages in pkg/ depend on other packages in pkg/, and our linuxkit config ymls depend on all of the packages in pkg/. These have image tags generated by using the git tree hash, which changes with commits. Rather than having to modify the 6 or so files over and over again - and risking missing changes - we do the following:

1. Calculate the expected tags of the docker images in pkg/ with no other dependencies in pkg/
2. Generate the Dockerfile for those docker images in pkg/ with dependencies on other packages in pkg/ based on the output of the previous step
3. Generate the rootfs.yml for the OS image based on the outputs of the two previous steps

The actual trees of dependencies is pretty small:

• most packages in pkg/ are completely independent
• pkg/xen-tools depends on pkg/uefi
• pkg/qrxec-lib depends on pkg/xen-tools
• pkg/qrexec-dom0 depends on pkg/qrexec-lib and pkg/xen-tools
• rootfs.yml depends on all pkg/

The challenges with the way it is currently structured are:

• inconsistency between repos. For example, I can build pkg/strongswan by doing lkt pkg build pkg/strongswan or even just cd pkg/strongswan; docker build ., but I have an extra step I need to do for xen-tools.
• inability to just build a package on its own. If I tried the above, it wouldn't even work, as there is no Dockerfile, just a Dockerfile.in
• inability to separate packages out into separate repos in the future, if desired
• missing source information in version control. We can generate it, but we don't actually keep it.

Here are some potential alternative paths.

hashes in files

One approach is to replace the Dockerfile.in and rootfs.yml.in with just Dockerfile and rootfs.yml which have the actual hashes in them. Maintaining these would be a pain, so we would need to have a tool that updates the files in place. This resolves most of the above issues. The approach would look something like:

1. Just build
2. If I change something, I need to run the tool (a changed parse-pkgs.sh or similar) that would update the files that need updating.
3. Check the updated files into version control, if desired

This is the approach that linuxkit itself has been using for a few years. It has pros and cons - sometimes you forget to update, but that can be handled by a Makefile. Most importantly, you know exactly where you stand. Because the Dockerfile and rootfs.yml are in version control, not just the *.in files, you know immediately if something has changed.

build-args

docker build supports --build-args; linuxkit pkg and linuxkit build do not. We could extend it so that it does, and the builds would be something like the following:

Dockerfile for xen-tools, checked into git:

ARG UEFI_TAG
FROM lfedge/eve-uefi:${UEFI_TAG} as uefi-build

And build it as:

docker build --build-arg UEFI_TAG=abcd1234 ...
# OR
linuxkit pkg build --build-arg UEFI_TAG=abcd1234 ...

We would need to generate the --build-arg options using some variant on parse-pkgs.sh, but again it is doable.

This doesn't get everything in version control, but does simplify the build process

build-args with root build.yml

This is a combination of the two previous. Rather than generating the build-args on the fly, we keep the build-args in a single file in key=value or yml format in the root of this repo. We then have the build source it. We regenerate the file as needed.

For discussion @rvs @eriknordmark @kalyan-nidumolu , with whom I have discussed this to some degree, and anyone else interested.