Cluster-API: upstream support for most common providers

[t-lo]

Support Flatcar in Cluster-API

Tracked Issues and PRs Foundational

• [x] Kubeadm Bootstrap provider ignition support: https://github.com/kubernetes-sigs/cluster-api/pull/4172
  • Related / dataurl dependency: https://github.com/vincent-petithory/dataurl/issues/7
• [X] Image Builder support for Flatcar: https://github.com/kubernetes-sigs/image-builder/pull/248
• [x] https://github.com/kubernetes-sigs/image-builder/pull/701

Platforms / Providers

• AWS: https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1979
  • Blocked by Ignition bootstrap PR
  • [x] Support ignition for bootstrapping: https://github.com/kubernetes-sigs/cluster-api-provider-aws/pull/2271
• Azure: https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/629 ( downstream tracking issue )
  • Blocked by Ignition bootstrap PR
  • [ ] Add Flatcar Container Linux template to CAPZ: https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/1729
  • [x] Support for Shared Image Gallery images requiring a Plan : https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/1463 , https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/1469
  • [x] image-builder: support for Azure SIG flatcar image https://github.com/kubernetes-sigs/image-builder/pull/652
  • [x] TODO: update azure go library in Mantle to support unmanaged disk images. Fallout of https://github.com/kinvolk/Flatcar/issues/417
• Equinix Metal:
  • [ ] Upstream PR for v1alpha4/v1beta1 support https://github.com/kubernetes-sigs/cluster-api-provider-packet/pull/289
  • [ ] WIP: template + metadata injection work needed
• Metal3:
  • [X] Support for Flatcar images: https://github.com/metal3-io/metal3-dev-env/pull/457
  • [ ] CAPI provider supports Flatcar: https://github.com/metal3-io/cluster-api-provider-metal3
• [ ] Tinkerbell
• vSphere
  • [ ] https://github.com/kubernetes-sigs/image-builder/pull/817 VSphere Image builder support for Flatcar (blocked by below)
  • [ ] https://github.com/kubernetes-sigs/image-builder/pull/827
  • [ ] https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/pull/1429 Ignition support for VSphere
• GCP
  • Tracking issue: https://github.com/kubernetes-sigs/cluster-api-provider-gcp/issues/540

Tests

• https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api-provider-aws
• https://prow.k8s.io/?repo=kubernetes-sigs%2Fcluster-api-provider-aws

SIG cluster lifecycle meetings calendar: https://calendar.google.com/calendar/u/0/[email protected]

Follow-up work (not checkboxed since it does not contribute to the roandmap item)

1. Publish a blog post on our changes and additions to CAPI to make Ignition and Flatcar work.
2. Integrate into the release process and publish Flatcar CAPI images on Azure and AWS.
3. AWS provider: For Ignition support to graduate out of experimental phase we need to abstract out S3 support into a separate CRD, ref: issue comment.

[t-lo]

@dongsupark The Azure image-builder PR is still failing a test (containerd version detection), can you help? https://github.com/kubernetes-sigs/image-builder/pull/652

[surajssd]

@t-lo I don't have access to edit the issue comment. But for Ignition support to graduate out of experimental phase we need to abstract out S3 support into a separate CRD, ref: issue comment.

[till]

I would contribute openstack related support, if there is any interest?

[t-lo]

@till There absolutely is! Please pardon the late reply. We're basing the support for individual providers on our work to support Ignition config as an alternative to cloud-init in the kubeadm bootstrap provider; that's a good point to start.

[till]

@t-lo No worries.

Is this ultimately replacing the work in lokomotive? I hacked around in it to add Openstack, but wasn't able to gain any quick wins.

[t-lo]

Is this ultimately replacing the work in lokomotive? I hacked around in it to add Openstack, but wasn't able to gain any quick wins.

It's not related to Lokomotive; CAPI and Lokomotive have different use cases.

[t-lo]

@till The PR to add Ignition support to kubeadm was merged and will be part of the CAPI v1.1 release next year. That's a good point in time to get started with Openstack support ;) Please feel free to ping us here if you have questions.

[johananl]

I'm looking into vSphere support.

[ericgraf]

@t-lo If possible we would like to have support for GCP and OpenStack added to the supported provider list.

[t-lo]

@ericgraf Ignition support was added in the v1.1 CAPI release yesterday, making it significantly easier to add support for more providers. While we currently don't have plans to work on GCP or OpenStack please feel free to pick this up, I'll happily update the tracking issue to reflect the work. Please do not hesitate to reach out to us for pointers and for help - the maintainers are available all day via https://app.element.io/#/room/#flatcar:matrix.org.

[srm09]

Adding @johananl 's PR for vSphere support here: https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/pull/1429

[AverageMarcus]

@t-lo FYI I've opened an issue on CAPG to request Flatcar support - https://github.com/kubernetes-sigs/cluster-api-provider-gcp/issues/540

[t-lo]

@AverageMarcus Thank you! I've added the tracking issue to the summary.

[t-lo]

@AverageMarcus Would you be interested to work on CAPG support? The team is supportive of the idea but currently pursues other priorities. We can definitely ramp you up on existing CAPI work and support you in the process.

[AverageMarcus]

I can't currently as have other work with higher priorities but possibly in the future once this becomes more of a priority for us. I mainly wanted to ensure it was tracked on CAPG so there was at least an awareness of wanting this.

[invidian]

Happy to ping everyone informing that CAPA support has been merged today :tada:

[kopiczko]

@t-lo I opened an issue for openstack https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues/1261. I'm working on this currently.

[kopiczko]

There will be changes required to the image-builder as well. Currently I'm basing my work on the QEMU image but grub.cfg requires modification. There is also a WIP PR to fix ongoing issues with the build https://github.com/kubernetes-sigs/image-builder/pull/907

[t-lo]

There will be changes required to the image-builder as well. Currently I'm basing my work on the QEMU image but grub.cfg requires modification. There is also a WIP PR to fix ongoing issues with the build kubernetes-sigs/image-builder#907

Please feel free to reach out if you feel stuck. @pothos and @dongsupark are good peers for Flatcar related image builder issues, and @johananl did quite some work on image builder lately, too.

[t-lo]

We'll soon retire this epic tracking issue in favour of our new ClusterAPI tab in our Projects board: https://github.com/orgs/flatcar/projects/7/views/14

[johananl]

Everything was migrated to the board 🎉. Closing.