community icon indicating copy to clipboard operation
community copied to clipboard

Kubeflow Graduation Proposal (2025)

Open castrojo opened this issue 7 months ago • 8 comments

The template and process for graduation is updated so it makes more sense to post a clean issue. There are still things spread across issues so let's make this a working doc: Here are the steps for Kubeflow:

Self Assessment

  • [x] Complete the security self assessment
  • Tracking Issue: https://github.com/cncf/tag-security/issues/1079
  • [x] Apply for third party audit as soon as completed (ASAP!)

Apply for Graduation

  • [ ] Complete the Graduation template (I'll post it in a comment below)
    • [ ] Replace Engineering Principles steps with the general technical review
    • Tracking Issue: https://github.com/kubeflow/community/issues/859
    • [ ] Submit application to the TOC once completed
    • [ ] Begin reaching out to adopters for interviews soon, need five.
      • This is cat herding so much be done early, and some orgs need to ask permission, etc. Interview questions

The TOC uses this criteria while reviewing our application.

Document Process in a Kubeflow repository

  • [ ] Commit to Security Assessment to a kubeflow repo once completed
  • [ ] Commit general technical review to a kubeflow repo once completed

Replaces: https://github.com/kubeflow/community/issues/655

castrojo avatar May 19 '25 14:05 castrojo

---
name: Project Graduation Application
about: This template provides the project with a framework to inform the TOC of their conformance to the Graduation Level Criteria.
title: "[Graduation] $PROJECT Graduation Application"
labels: graduation
---

Review Project Moving Level Evaluation

  • [x] I have reviewed the TOC's moving level readiness triage guide, ensured the criteria for my project are met before opening this issue, and understand that unmet criteria will result in the project's application being closed.

Kubeflow Graduation Application

v1.6 This template provides the project with a framework to inform the TOC of their conformance to the Graduation Level Criteria.

Project Repo(s): https://github.com/kubeflow

Project Site: https://www.kubeflow.org/

Sub-Projects:

Communication: CNCF Slack: https://www.kubeflow.org/docs/about/community/#slack-channels

Project point of contact: [email protected]

In progress

Graduation Criteria Summary for Kubeflow

Application Level Assertion

  • [x] This project is currently Incubating, accepted on 2023-07-23, and applying to Graduate.

Adoption Assertion

The project has been adopted by the following organizations in a testing and integration or production capacity:

Kubeflow is used by thousands of organizations worldwide with over 9.6M downloads per month: https://insights.linuxfoundation.org/project/kubeflow/popularity?timeRange=past365days&widget=package-downloads&start=2024-07-16&end=2025-07-16

We maintain a best-effort list of adopters for Kubeflow Projects:

  • https://github.com/kubeflow/community/blob/master/ADOPTERS.md
  • https://github.com/kubeflow/spark-operator/blob/master/ADOPTERS.md
  • https://github.com/kubeflow/notebooks/blob/main/ADOPTERS.md
  • https://github.com/kubeflow/trainer/blob/master/ADOPTERS.md
  • https://github.com/kubeflow/katib/blob/master/ADOPTERS.md
  • https://github.com/kubeflow/model-registry/blob/main/ADOPTERS.md
  • https://github.com/kubeflow/pipelines/blob/master/ADOPTERS.md

Application Process Principles

Suggested

N/A

Required

  • [x] Engage with the domain specific TAG(s) to increase awareness through a presentation or completing a General Technical Review.

    • [x] TAG provides insight/recommendation of the project in the context of the landscape
      • TAG Runtime - Presentation 21-01-2021: https://youtu.be/S6N8ARZZcGs
      • TAG Runtime and TOC AI Initiatives - Presentation 24-11-2024: https://youtu.be/u4Mf3Jh8v2E?t=2243
  • [x] All project metadata and resources are vendor-neutral.

  • Kubeflow governance model limit representation of organizations in steering committee and working groups

  • All project communication is vendor neutral

  • More info: https://www.kubeflow.org/docs/about/governance/

  • More info: https://github.com/kubeflow/community/tree/master/proposals/645-kubeflow-steering-committee-election#changes-to-take-effect-in-2025-election-and-beyond

  • [x] Review and acknowledgement of expectations for graduated projects and requirements for moving forward through the CNCF Maturity levels.

    • [x] Met during Project's application on July 2025

Completion of this due diligence document, resolution of concerns raised, and presented for public comment satisifies the Due Diligence Review criteria.

  • [x] Additional documentation as appropriate for project type, e.g.: installation documentation, end user documentation, reference implementation and/or code samples.

    • Kubeflow docs: https://www.kubeflow.org/docs/
    • Introduction docs: https://www.kubeflow.org/docs/started/introduction/
    • Installation docs: https://www.kubeflow.org/docs/started/installing-kubeflow/
    • Community: https://www.kubeflow.org/docs/about/community/
    • Due Diligence for Kubeflow Incubation

Governance and Maintainers

Note: this section may be augmented by the completion of a Governance Review from TAG Contributor Strategy.

Suggested

  • [x] Governance has continuously been iterated upon by the project as a result of their experience applying it, with the governance history demonstrating evolution of maturity alongside the project's maturity evolution.

  • Governance revised multiple times started from 2019 with WG formation: https://github.com/kubeflow/community/blob/master/wgs/overview.md

  • As of 2023 Kubeflow community performed two elections for Kubeflow Steering Committee (KSC): https://github.com/kubeflow/community/tree/master/elections

  • As of 2024 KSC enforced rule to limit number sits for one organization: https://github.com/kubeflow/community/tree/master/proposals/645-kubeflow-steering-committee-election#changes-to-take-effect-in-2025-election-and-beyond

Required

  • [x] Clear and discoverable project governance documentation.

  • https://www.kubeflow.org/docs/about/governance/

  • https://github.com/kubeflow/community

  • [x] Governance is up to date with actual project activities, including any meetings, elections, leadership, or approval processes.

    • KSC charter is updated: https://github.com/kubeflow/community/blob/master/KUBEFLOW-STEERING-COMMITTEE.md#charter
    • Elections are tracked here: https://github.com/kubeflow/community/tree/master/elections
    • WG meetings are up-to-date: https://www.kubeflow.org/docs/about/community/#list-of-available-meetings
  • [x] Governance clearly documents vendor-neutrality of project direction.

    • Maximum of one organization can be on KSC: https://github.com/kubeflow/community/tree/master/proposals/645-kubeflow-steering-committee-election#changes-to-take-effect-in-2025-election-and-beyond
    • Maximum of one organization can be on KOC: https://github.com/kubeflow/community/blob/master/KUBEFLOW-OUTREACH-COMMITTEE.md#limitations-on-company-representation
  • [x] Document how the project makes decisions on leadership roles, contribution acceptance, requests to the CNCF, and changes to governance or project goals.

  • [x] Document how role, function-based members, or sub-teams are assigned, onboarded, and removed for specific teams (example: Security Response Committee).

  • [x] Document complete list of current maintainers, including names, contact information, domain of responsibility, and affiliation.

  • [x] A number of active maintainers which is appropriate to the size and scope of the project.

  • [x] Document a complete maintainer lifecycle process (including roles, onboarding, offboarding, and emeritus status).

    • It is explained here: https://www.kubeflow.org/docs/about/membership/
  • [x] Demonstrate usage of the maintainer lifecycle with outcomes, either through the addition or replacement of maintainers as project events have required.

    • Project leads constantly add and remove maintainers. For example
      • Kubeflow Pipelines: https://github.com/kubeflow/pipelines/pull/12059
      • Kubeflow Trainer: https://github.com/kubeflow/trainer/pull/2659
      • Kubeflow Model Registry: https://github.com/kubeflow/model-registry/pull/1153
  • [x] Project maintainers from at least 2 organizations that demonstrates survivability..

    • Maintainers cover > 10 organizations
  • [x] Code and Doc ownership in Github and elsewhere matches documented governance roles.

    • We use prow and tide to define ownership in GitHub. Access to the repos are given by using these YAML.
  • [x] Document adoption of the CNCF Code of Conduct

    • Kubeflow follows CNCF Code of Conduct: https://www.kubeflow.org/docs/about/contributing/#follow-the-code-of-conduct
  • [x] CNCF Code of Conduct is cross-linked from other governance documents.

    • https://www.kubeflow.org/docs/about/governance/#1-cncf
  • [x] All subprojects, if any, are listed.

    • Kubeflow Spark Operator
    • Kubeflow Notebooks
    • Kubeflow Trainer
    • Kubeflow Katib
    • Kubeflow Model Registry
    • Kubeflow Pipelines
  • [x] If the project has subprojects: subproject leadership, contribution, maturity status documented, including add/remove process.

    • Kubeflow projects are controlled by Kubeflow WGs. The add/remove process is defined here: https://github.com/kubeflow/community/blob/master/how-to/join_kubeflow_ecosystem.md

Contributors and Community

Note: this section may be augmented by the completion of a Governance Review from TAG Contributor Strategy.

Suggested

Required

  • [x] Clearly defined and discoverable process to submit issues or changes.

    • https://www.kubeflow.org/docs/about/contributing/#starter-issues
    • KEP process: https://github.com/kubeflow/community/tree/master/proposals
  • [x] Project must have, and document, at least one public communications channel for users and/or contributors.

    • Kubeflow uses CNCF slack: https://www.kubeflow.org/docs/about/community/#slack-channels
    • #kubeflow-spark-operator
    • #kubeflow-notebooks
    • #kubeflow-trainer
    • #kubeflow-katib
    • #kubeflow-model-registry
    • #kubeflow-pipelines
  • [x] List and document all project communication channels, including subprojects (mail list/slack/etc.). List any non-public communications channels and what their special purpose is.

    • Kubeflow projects have a dedicated CNCF Slack channel #kubeflow-contributors used for developer communication.
    • Kubeflow KSC has private mailing list for maintainers, users asks, and for security reports: [email protected]
    • Kubeflow users this Slack channel for announcements: #kubeflow-announcements
    • Kubeflow has dedicated mailing: [email protected]
    • Kubeflow also uses various social media: LinkedIn: https://www.linkedin.com/company/kubeflow/, X: https://x.com/kubeflow, BlueSky: https://bsky.app/profile/kubefloworg.bsky.social, YouTube Channel: https://www.youtube.com/@Kubeflow, and https://www.youtube.com/@KubeflowCommunity
  • [x] Up-to-date public meeting schedulers and/or integration with CNCF calendar.

    • Kubeflow calendar for WG and community meetings: https://www.kubeflow.org/docs/about/community/#list-of-available-meetings
  • [x] Documentation of how to contribute, with increasing detail as the project matures.

    • Each project also provides new contributors specific guidance on their GitHub repos:
    • https://github.com/kubeflow/pipelines/blob/master/CONTRIBUTING.md
    • https://github.com/kubeflow/trainer/blob/master/CONTRIBUTING.md#best-practices
    • https://github.com/kubeflow/katib/blob/master/CONTRIBUTING.md
    • https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md
    • https://github.com/kubeflow/spark-operator/blob/master/CONTRIBUTING.md
    • https://github.com/kubeflow/notebooks/blob/main/CONTRIBUTING.md
  • [x] Demonstrate contributor activity and recruitment.

    • Kubeflow projects have ~ 1,500 active contributors from ~ 280 organizations: https://insights.linuxfoundation.org/project/kubeflow/contributors?timeRange=past365days&start=2024-07-15&end=2025-07-15&widget=active-contributors
    • Active contributors have been promoted to project owners: https://github.com/kubeflow/trainer/pull/2659
    • Kubeflow DevStat

Engineering Principles

This section is covered by the General Technical Review, tracked here: https://github.com/kubeflow/community/issues/859

(Link to original template for reference

Kubeflow General Technical Review: https://docs.google.com/document/d/15CZtkk3x-YIUaNnaRzIZaIKrfDfT6is_PAlRDIoBKgQ/edit?tab=t.0

Security

Note: this section may be augmented by a joint-assessment performed by TAG Security.

Suggested

  • [x] Achieving OpenSSF Best Practices silver or gold badge. Currently two Kubeflow projects achieve it.

    • Kubeflow Katib: https://www.bestpractices.dev/projects/9941
    • Kubeflow Trainer: https://www.bestpractices.dev/projects/10435

Required

  • [x] Clearly defined and discoverable process to report security issues.

    • Kubeflow Spark Operator security policy: https://github.com/kubeflow/spark-operator/blob/master/SECURITY.md
    • Kubeflow Notebooks security policy: https://github.com/kubeflow/notebooks/blob/master/SECURITY.md
    • Kubeflow Trainer security policy: https://github.com/kubeflow/trainer/blob/master/SECURITY.md
    • Kubeflow Katib security policy: https://github.com/kubeflow/katib/blob/master/SECURITY.md
    • Kubeflow Model Registry security policy: https://github.com/kubeflow/model-registry/blob/main/SECURITY.md
    • Kubeflow Pipelines security policy: https://github.com/kubeflow/pipelines/blob/master/SECURITY.md
  • [x] Enforcing Access Control Rules to secure the code base against attacks (Example: two factor authentication enforcement, and/or use of ACL tools.)

    • ACLs are required to be part of Kubeflow GitHub org.
  • [x] Document assignment of security response roles and how reports are handled.

    • Kubeflow Spark Operator security policy: https://github.com/kubeflow/spark-operator/blob/master/SECURITY.md
    • Kubeflow Notebooks security policy: https://github.com/kubeflow/notebooks/blob/master/SECURITY.md
    • Kubeflow Trainer security policy: https://github.com/kubeflow/trainer/blob/master/SECURITY.md
    • Kubeflow Katib security policy: https://github.com/kubeflow/katib/blob/master/SECURITY.md
    • Kubeflow Model Registry security policy: https://github.com/kubeflow/model-registry/blob/main/SECURITY.md
    • Kubeflow Pipelines security policy: https://github.com/kubeflow/pipelines/blob/master/SECURITY.md
  • [x] Document Security Self-Assessment.

    Kubeflow Security Self-Assessment: https://github.com/kubeflow/community/blob/master/security/self-assessment.md

  • [ ] Third Party Security Review.

    • [ ] Moderate and low findings from the Third Party Security Review are planned/tracked for resolution as well as overall thematic findings, such as: improving project contribution guide providing a PR review guide to look for memory leaks and other vulnerabilities the project may be susceptible to by design or language choice ensuring adequate test coverage on all PRs.

      • Kubeflow third party Security Review is in progress.
  • [x] Achieve the Open Source Security Foundation (OpenSSF) Best Practices passing badge.

    • Kubeflow Spark Operator: https://www.bestpractices.dev/en/projects/10524
    • Kubeflow Notebooks: https://www.bestpractices.dev/en/projects/9942
    • Kubeflow Katib: https://www.bestpractices.dev/projects/9941
    • Kubeflow Trainer: https://www.bestpractices.dev/projects/10435
    • Kubeflow Model Registry: https://www.bestpractices.dev/en/projects/9937
    • Kubeflow Pipelines: https://www.bestpractices.dev/en/projects/9938

Ecosystem

Suggested

N/A

Required

  • [x] Publicly documented list of adopters, which may indicate their adoption level (dev/trialing, prod, etc.)

    • https://github.com/kubeflow/community/blob/master/ADOPTERS.md
    • https://github.com/kubeflow/spark-operator/blob/master/ADOPTERS.md
    • https://github.com/kubeflow/notebooks/blob/main/ADOPTERS.md
    • https://github.com/kubeflow/trainer/blob/master/ADOPTERS.md
    • https://github.com/kubeflow/katib/blob/master/ADOPTERS.md
    • https://github.com/kubeflow/model-registry/blob/main/ADOPTERS.md
    • https://github.com/kubeflow/pipelines/blob/master/ADOPTERS.md
  • [x] Used in appropriate capacity by at least 3 independent + indirect/direct adopters, (these are not required to be in the publicly documented list of adopters)

  • The project provided the TOC with a list of adopters for verification of use of the project at the level expected.

The project provided the TOC with a list of adopters for verification of use of the project at the level expected, i.e. production use for graduation, dev/test for incubation.

  • [ ] TOC verification of adopters.

    • Pending TOC

Refer to the Adoption portion of this document.

  • [x] Clearly documented integrations and/or compatibility with other CNCF projects as well as non-CNCF projects.

    • Kubeflow projects use and integrate with many projects from CNCF ecosystem including Kubernetes, Argo Workflow, Istio, Cert-Manager, KServe, JobSet, Kueue, Dex, Knative, Volcano, and others. Additionally, Kubeflow uses many non-CNCF projects from AI/ML ecosystem: PyTorch, DeepSpeed, JAX, Jupyter, Apache Spark, Horovod, MLX, XGBoost, HuggingFace, and many others.

Adoption

Adopter 1 - Slalom/Healthcare

September 2022

Adopter 2 - Capital One/Finance

September 2019

Adopter 3 - CERN/Academic Institutions

September 2019

castrojo avatar May 19 '25 14:05 castrojo

/area cncf-graduation

andreyvelich avatar May 27 '25 20:05 andreyvelich

per slack, 3 minor comments?

Image just formatting typo `KSC elections are explained (in this doc)[https://github.com/kubeflow/community/tree/master/proposals/645-kubeflow-steering-committee-election]` not proper md.

Are we sure that link "WG chairs and leads" reflects correctly? I believe that owners file following membership guideline in each repo is more accurate.

"Suggested: Achieving OpenSSF Best Practices silver or gold badge." 👉 I think this checkbox for suggested should be left UN-checked :)

tarilabs avatar Jul 18 '25 07:07 tarilabs

b.com/kubeflow/community

Great catch @tarilabs!

Are we sure that

Actually, this YAML should represent the updated information for WG leads. We should update it if that is not accurate. I will also add the OWNERs file for each project. Let me also add this Maintainer file: https://github.com/kubeflow/community/blob/master/MAINTAINERS.md

think this checkbox for suggested should be left UN-checked :)

Good point.

andreyvelich avatar Jul 18 '25 13:07 andreyvelich

@andreyvelich in the examples of

Project leads constantly add and remove maintainers

could you also kindly add this PR? https://github.com/kubeflow/model-registry/pull/1153

tarilabs avatar Jul 19 '25 05:07 tarilabs

@andreyvelich in the examples of

Project leads constantly add and remove maintainers

could you also kindly add this PR? kubeflow/model-registry#1153

Done.

andreyvelich avatar Jul 21 '25 18:07 andreyvelich

We created the Kubeflow CNCF Graduation application: https://github.com/cncf/toc/issues/1861

andreyvelich avatar Sep 09 '25 14:09 andreyvelich