WIP: build: Add support for "variants"

[cgwalters]

My immediate goal here is to change https://github.com/cgwalters/centos-coreos-stream-config into something more like "openshift/coreos-config" that can be built in multiple ways.

We'd have rhcos and cscos as "build variants". And actually while we're here potentially rhbasecos which would be "just stuff from RHEL".

Another use case for this is the "alternative desktops" for Silverblue.

And yet another use case is something like a "debug" build for FCOS where e.g. we swap in kernel-debug and potentially turn on other userspace debugging too.

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [cgwalters]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[cgwalters]

And another use case: I've been thinking lately that it would simplify a whole bunch of things if for RHCOS we split up "bootimage build" and "oscontainer build'. Once the bootstrap node pivots then in fact I think we could have the bootimage just be RHEL - we keep kubelet and the other bits in the oscontainer. This would make OCP/RHCOS operate the same way OKD/FCOS does.

It'd also solve various "source of truth" problems where we push to both a registry and S3 - for the oscontainer the registry should be canonical.

[cgwalters]

Once the bootstrap node pivots then in fact I think we could have the bootimage just be RHEL - we keep kubelet and the other bits in the oscontainer. This would make OCP/RHCOS operate the same way OKD/FCOS does.

(Though this conflicts somewhat with the live ISO containing kubelet, but there's no reason we can't do live layering eventually)

[cgwalters]

Yet another obvious use case: rather than having e.g. git branches for e.g. rhel-8.1 and rhel-8.2, keep them in the same tree and make them build variants.

This gets into a whole interesting topic around the representation of the fedora-coreos-config branches and whether we should instead just use git master and have testing-devel/ and next/ etc. as subdirectories. Adding this feature doesn't require switching obviously, but it's at least interesting to think about. I am pretty sure it'll work well for RHCOS at least.

[jlebon]

And yet another use case is something like a "debug" build for FCOS where e.g. we swap in kernel-debug and potentially turn on other userspace debugging too.

Hmm, how would this work? rpm-ostree treefile merges extend lists, which means that e.g. to swap out kernel for kernel-debug would require changing the way we inherit things since it's part of bootable-rpm-ostree.yaml which is at the bottom of the inclusion tree.

[openshift-ci-robot]

@cgwalters: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci[bot]]

@cgwalters: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/images	befd3b0a2fc01426c17c1e2f634bd9d0a72dc8bf	link	true	/test images
ci/prow/rhcos	befd3b0a2fc01426c17c1e2f634bd9d0a72dc8bf	link	true	/test rhcos

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[travier]

Closing this one as we have merged variant support in https://github.com/coreos/coreos-assembler/pull/2934