Enhancement Description

• One-line enhancement description (can be used as a release note): Speed up container startup by mounting volumes with the correct SELInux label instead of changing each file on the volumes recursively.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1710-selinux-relabeling

• Primary contact (assignee): @jsafrane

• Responsible SIGs: sig-storage, sig-node

The KEP describes 3 phases / 3 feature gates.

SELinuxMountReadWriteOncePod:

• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.24
  • Beta release target (x.y): 1.27
  • Stable release target (x.y): 1.34
• [x] Alpha
  • [x] KEP (k/enhancements) update PR(s):
  • [x] Code (k/k) update PR(s):
  • [x] Docs (k/website) update PR(s):
• [x] Beta
  • [x] KEP (k/enhancements) update PR(s):
    • https://github.com/kubernetes/enhancements/pull/3797
  • [x] Code (k/k) update PR(s):
    • https://github.com/kubernetes/kubernetes/pull/116425
  • [x] Docs (k/website) update(s):
    • https://github.com/kubernetes/website/pull/39836

SELinuxChangePolicy

• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.32
  • Beta release target (x.y): 1.33-34
  • Stable release target (x.y): ?
• [x] Alpha
  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/4843
  • [ ] Code (k/k) update PR(s):
    • new/updated test jobs:
  • [ ] Docs (k/website) update PR(s):
• [ ] Beta
  • [ ] KEP (k/enhancements) update PR(s):
  • [ ] Code (k/k) update PR(s):
  • [ ] Docs (k/website) update(s):

SELinuxMount

• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.30
  • Beta release target (x.y): 1.34-35
  • Stable release target (x.y): ?
• [x] Alpha
  • [x] KEP (k/enhancements) update PR(s): https://github.com/kubernetes/enhancements/pull/4436
  • [x] Code (k/k) update PR(s):
    • https://github.com/kubernetes/kubernetes/pull/123157
    • https://github.com/kubernetes/kubernetes/pull/123667
    • new/updated test jobs:
      • https://github.com/kubernetes/test-infra/pull/32125
      • https://github.com/kubernetes/test-infra/pull/32143
      • https://testgrid.k8s.io/google-aws#kops-aws-selinux
      • https://testgrid.k8s.io/google-aws#kops-aws-selinux-alpha
  • [x] Docs (k/website) update PR(s): https://github.com/kubernetes/website/pull/45280
• [ ] Beta
  • [ ] KEP (k/enhancements) update PR(s):
  • [ ] Code (k/k) update PR(s):
  • [ ] Docs (k/website) update(s):

/sig storage /sig node

Hey @jsafrane -- 1.19 Enhancements Lead here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

---

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

Hi @jsafrane,

Tomorrow, Tuesday May 19 EOD Pacific Time is Enhancements Freeze

Will this enhancement be part of the 1.19 release cycle?

@jsafrane -- Unfortunately, the deadline for the 1.19 Enhancement freeze has passed. For now, this is being removed from the milestone and 1.19 tracking sheet. If there is a need to get this in, please file an enhancement exception.

@palnabarun hey, we've just merged the KEP yesterday, at the last moment. I admit I did not pay attention to this enhancement issue and focused on the design. Do I really need an exception just to restore the milestone?

Do I really need an exception just to restore the milestone?

Yes, an exception would be needed. Here is the process on how to file and exception request.

@jsafrane -- Your exception request was approved. I have updated the tracking sheet accordingly.

/milestone v1.19

/stage alpha

Hi @jsafrane - My name is Zachary, 1.19 Docs shadow. Is this enhancement work planned for 1.19 and does it require any new docs (or modifications to existing docs)? If not, can you please update the 1.19 Enhancement Tracker Sheet, or let me know, I can do it for you :) If docs are required, just a friendly reminder that we are looking for a PR against k/website (branch dev-1.19) due by Friday, June 12, it can just be a placeholder PR at this time. Let me know if you have any questions!

@zestrells, yes, documentation will be needed. I can't edit the tracking sheet, can you please note it there?

Hey @jsafrane, I am with the enhancements team for the v1.19 release cycle as a shadow.

The code freeze deadline for the Enhancement is Thursday, June 25. I am checking in to see if there is any k/k PR that you have already opened for this enhancement and if so, would you be able to point me in the direction of the PR so that the same can be updated in the tracking sheet

Have a wonderful day. 🖖

Hi @jsafrane - Just a reminder that docs placeholder PR against dev-1.19 is due by June 12th. Does this enhancement require any changes to docs? If so, can you update here with a link to the PR once you have it in place? If not, please update the same, so that the tracking sheet can be updated accordingly. Thanks!

Hey @jsafrane, This is just a reminder that the code freeze for the enhancement is Thursday, June 25. I am checking in to see if there is any k/k PR that is already open against this enhancement that needs to be tracked.

Have a wonderful day. 🖖

API PR: https://github.com/kubernetes/kubernetes/pull/91838 WIP Docs: https://github.com/kubernetes/website/pull/21773

Hi, @jsafrane

This is a follow-up to the communication that went out to k-dev today. There has been a revision to the release schedule of v1.19 as follows.

Thursday, July 9th: Week 13 - Code Freeze
Thursday, July 16th: Week 14 - Docs must be completed and reviewed
Tuesday, August 25th: Week 20 - Kubernetes v1.19.0 released
Thursday, August 27th: Week 20 - Release Retrospective

You can find the revised Schedule in the sig-release Repo

Please let me know if you have any questions. 🖖

Hi @jsafrane ,

This is just a follow up to my earlier messages on the upcoming deadlines. The code freeze deadline is Thursday, July 9th EOD PST and I noticed that the k/k PRs are still in flight.

For the enhancement to be included into v1.19 this PR needs to be merged before the code freeze deadline.

Please refer to the Exception Process documentation in case if there is a need for one.

/milestone clear /milestone v1.20

/milestone v1.20

Hi @jsafrane !

Enhancements Lead here, do you intend to do work on this for alpha in 1.20?

Thanks! Kirsten

Hi @jsafrane

Following up is this going to be included in 1.20?

Thanks, Kirsten

Hi @jsafrane : Final Reminder: 1.20 Enhancements Freeze is October 6th. Could you let us know if you have plans for 1.20? Also, the current KEP needs a kep.yaml update along with a README.md update to correct the milestones.

Thanks. Kirsten

Hello, this feature won't be implemented in 1.20. We'd like to focus on the design in this release.

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen /remove-lifecycle rotten

@gnufied: Reopened this issue.

In response to this:

/reopen /remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Tested in k8s 1.20, CRI-O 1.20, changing fsGroupChangePolicy to OnRootMismatch significantly improves mount time, however volumes still take many minutes to be mounted due to this issue. I can see high CRI-O iops during the operation, and here is the events of pod:

 Warning  Failed          5m41s (x2 over 7m42s)  kubelet            Error: context deadline exceeded
  Warning  Failed          5m10s                  kubelet            Error: Kubelet may be retrying requests that are timing out in CRI-O due to system load: the requested container xxxx is now ready and will be provided to the kubelet on next retry: error reserving ctr name xxxx for id xxxx : name is reserved

This is a test volume with only 500K files which took 5 minutes to be mounted! (note we have IOPS qos (300 IO per second per volume) on our Cluster for volumes for stability reasons.)