coreos-assembler
coreos-assembler copied to clipboard
coreos
Add variant support
Revert "cmd-init: Support specifying a subdirectory"
Remove support for subdirectory in cosa init given that:
- we currently don't use this feature,
- we chose another path for multiple version support in openshift/os as using subdirectories lead to a lot of additionnal complexity.
See: https://github.com/openshift/os/pull/803
This reverts commit 4781e4663f8b55cbc451feded25bdbd276722d62.
cmd-init: Small Bash and styling fixes
cmd-*: Support config repos with multiple variants
Several variants may be provided from a single Git config repo:
- If no variant is explicitly selected then the default
manifest.yamlwill be used. - If there is no
manifest.yamlfile provided by default in the repo, then COSA will read thevariant_defaultfile to figure out which variant to use by default. - If you explicitly specify a variant as parameter then COSA will read
the corresponding
variant_$VARIANTfile and use that as default instead, overriding any default set in the repo.
See:
- https://github.com/openshift/os/pull/852
- https://github.com/openshift/os/pull/901
Reworks: 480c23969 init: Require specifying a config repository
schema: Add coreos-assembler.variant support
Make sure that we record which variant was build with a given config repo and commit.
I can also rework this PR to move the script into COSA and store the RHEL version into a file instead.
I've reworked the logic here to be declarative instead of relying on a script while keeping some of the properties that we have right now.
This introduces variant_foo config files that are used to store the name of the manifests corresponding to a variant. This enables us to have a default variant (variant_default) pointing to CentOS Stream, an rhcos variant (variant_rhcos) pointing to a given RHEL version and other explicitly versioned variants that point to a given RHEL/CentOS version (rhel-8.6 (variant_rhel-8.6), rhel-9.0 (variant_rhel-9.0), c9s (variant_c9s)).
- If no variant is explicitly selected then the default
manifest.yamlwill be used. This is to keep the current setup for FCOS. - If there is no
manifest.yamlfile provided by default in the repo, then COSA will read thevariant_defaultfile to figure out which variant to use by default. - If you explicitely specify a variant as parameter then COSA will read the corresponding
variant_$VARIANTfile and use that as default instead, overriding any default set in the repo.
The main advantage of doing things this way is that we can have the internal CI always use the rhcos variant and thus never need an update when we change the default version of RHCOS. Changing the default is entirely based on a single symlink in the openshift/os repo.
We can then switch the repo default to be SCOS and community contributors will get the CentOS Stream version by default.
We can then switch the repo default to be SCOS and community contributors will get the CentOS Stream version by default.
Hmm. Right, though I'm thinking we're trying to overload two things into one switch:
- distro: centos vs rhel
- major.minor: rhel 8.6 vs 9.0
I'd agree that it could make sense to logically switch the default distro to centos stream. But, we can have pipelines which want to build RHCOS set a flag (something like cosa init --variable=distro=rhel) that overrides that.
I think we want the default major.minor version is better hardcoded in openshift/os, but we also want something like cosa init --variable=major=9 or so so that pipelines can target rhel9.
OTOH, hopefully in a year or so we won't care at all about --variable=major=8 at all, which will help significantly with the matrix here.
(One thing I think we should also keep in mind here is...what if we switched fedora-coreos-config to not use git branches either? The lockfiles and CI there are oriented around it, but clearly this whole approach could work there too. It'd just be good to have the two work the same way. I'm not saying we should scope this in or block on it, but reiterating I think it would be a better end state to have the two work the same way)
I definitely want to make progress here and not block on a perfect design...but OTOH the semantics we choose here have a lot of ramifications and will get tied into various pipelines and be something everyone needs to understand, so that motivates some level of design discussion.
OTOH, I think we can change this later too without too much trouble, would just require some "ratcheting" across cosa and the pipelines.
The UX with this one would be:
# Get the default for the repo
$ cosa init https:/...../ repo.git
# Get the default for RHCOS
$ cosa init https:/...../ repo.git rhcos
# Get a specific version as explicitly asked
$ cosa init https:/...../ repo.git rhel-8.6
$ cosa init https:/...../ repo.git rhel-9.0
# Get the default for SCOS. Remember that we will switch SCOS to CentOS Stream 10 at some point too
$ cosa init https:/...../ repo.git scos
# Get a specific version as asked
$ cosa init https:/...../ repo.git c9s
All RHCOS pipelines will thus use the rhcos one. All SCOS pipeline will use the scos one. Other test jobs can be more explicit.
Note: CI is failing on an unrelated issue and is not testing that change currently.
This is better than a script, though I don't like that it dirties the config dir. And more generally, that it retains state in the config dir at all. Ideally state would remain in the cosa workdir instead and the config dir remains wholly described by the git SHA like today, which is an important property.
I like the variables idea from @cgwalters. I.e. we still have a single manifest.yaml, but we support overriding default variables there, and the manifest conditionally includes things based on those. We then keep the values in src/vars.yaml alongside src/config.
It needs some more fleshing out though, because there's also image.yaml and extensions.yaml involved which don't support this. Maybe we can have a special variant variable which is used in the treefiles, but also for finding e.g. image-${variant}.yaml?
Anyway, short-term I'd still personally prefer the subdirs approach. The cosa patch for it is not that large. But I'm OK with this if we really want. Won't mention this again now because I don't want to block this!
Though looking at this patch, I think we could enhance it slightly to avoid the few symlinks it does by keeping the variant active in e.g. src/variant or something? And then e.g. buildextend-extensions knows to read that file to get the path to extensions.yaml to hand over to rpm-ostree?
Ideally state would remain in the cosa workdir instead and the config dir remains wholly described by the git SHA like today, which is an important property.
But after this lands, note the system state won't be fully described by the git sha; imagine that one wants to do cosa buildprep on top of a previous build for e.g. RHCOS. What will happen now is that the user will also need to manually cosa init to the target variant.
I think what we want is to have this information inside meta.json for example, and then cosa buildprep would also use it.
We've discussed that today and I'll update this PR to be closer to https://github.com/coreos/coreos-assembler/pull/1459 and what's in the comments above (recording the variant in the meta.json).
I've updated this one with the feedback from our design discussions. I'm currently testing it to make sure we don't miss a corner case as the CI here does not test that.
Right now the selected variant is stored directly in src/osver but maybe this should be a JSON document?
On top of this one, I need to add support for kola to read different denylist files or to use a key to filter by variant.
And I should change the code to store the variant as requested on the command line and not as resolved so that if you select RHCOS then you get auto-updated to the latest version as you git pull.
Updated to fix review comments. Currently testing all RHCOS/SCOS configurations to make sure this works.
This is still missing a solution for kola tests.
I've significantly simplified this one to only include what's needed for variant support.
Should we support --variant default being equivalent to not passing a variant? This could be useful in scripting where the variant passed to cosa init is a variable.
Not a blocker of course. Good to do in a follow-up.
(Still planning to do a last round of testing but haven't had the time yet).
I've tested that with 4 variants and it worked so far. Will merge on Monday to be able to do follow-up changes as needed.