Switch to cgroups v2 for all images

[dims]

trafficstars

The eco system is moving towards cgroups v2 instead of cgroups v1. Kubelet, containerd, runc, cadvisor etc has been updated to work with cgroups v2. Docker Desktop has switched already to cgroups v2 as well by default.

Can we please switch our default images to cgroups v2? thanks!

context: https://github.com/kubernetes/kubernetes/issues/108028

[codenrhoden]

Thanks for bringing this up, @dims. Reading through the related issue, seems like changing the default is reasonable, but we definitely want people to still be able to use v1 if they want. Additionally, we'll have to actually check if all the OS targets we build are able to use v2.

[dims]

@codenrhoden indeed. the title was click-bait-y :) intended to be aspirational for sure!

[dims]

@codenrhoden do we want to check if this can be done in 1.25 time frame?

[codenrhoden]

Hi @dims,

We discussed this a bit at this week's office hours, and it was agreed that we should add the ability to enable v2 (since there's really no option for it right now). But there were concerns that some OS's have documented issues with specific workloads on v2, so perhaps a more conservative approach is called for. We can make v2 an option, keeping v1 as the default, OR we can make v2 a default to gather more data (with the option, to build with v1 instead), but we specifically set v1 as default still for the Flatcar, which is the OS that has the most well-known workload performance degradation with v2.

[dims]

@codenrhoden works!

cc @endocrimes

[t-lo]

@dims To elaborate on our conversation in the image builder office hours, the Flatcar project switched to v2 by default in November 2021 (https://www.flatcar.org/releases/#release-2983.2.0) after the feature spent a few months in Alpha and Beta.

While cgroupsv2 is well integrated with Flatcar and works fine in principle, many of our users ran into issues with their workloads (legacy Java in particular) as well as with common Kubernetes components they were using (CAdvisor, Cilium, kops, etc). We track issues know to us here: https://www.flatcar.org/releases/#release-2983.2.0 . Based on significant user feedback we subsequently introduced a provisioning option to deploy Flatcar with v1: https://github.com/flatcar-linux/coreos-overlay/pull/1666

We'll be fine with Flatcar and image builder in any case since we can simply instrument Flatcar to run with v1; just speaking up here because considering the above experience I'd expect some friction down the road when switching core components to cgroupsv2.

[dims]

@t-lo yes, it's going to be .... not easy!

[t-lo]

Precisely what I wanted to express 😅

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.