website icon indicating copy to clipboard operation
website copied to clipboard

Update Kubeflow Installation with Standalone Mode

Open andreyvelich opened this issue 1 year ago • 19 comments

As we discussed multiple times before, we want to explain our users that Kubeflow components can be deployed:

  • in Standalone mode.
  • in Kubeflow Platform for end-to-end AI/ML experience.

In this PR I made changes to the installation section: I removed What is Kubeflow? section from this page since we already have this section here: https://www.kubeflow.org/docs/started/introduction/#what-is-kubeflow, we can update it if you think it is out-of-date.

I made these sections:

  • Install Kubeflow Components Standalone
  • Install Kubeflow Platform from Raw Manifests
  • Install Kubeflow Platform from Packaged Distributions

From my point of view this is logically correct, for users to learn about Kubeflow ecosystem. Firstly, users will explore the value of individual Kubeflow components. Secondly, if users need end-to-end AI/ML experience with all Kubeflow components, they will explore the Raw Manifests and Package Distributions installation.

What do you think about it ?

I am happy to work on the language to make it less confusing for our users if you have other ideas on how we can improve it.

/hold for review

/assign @kubeflow/wg-notebooks-leads @kubeflow/wg-pipeline-leads @kubeflow/wg-training-leads @kubeflow/wg-data-leads @kubeflow/release-team @kubeflow/kubeflow-steering-committee @kubeflow/wg-deployment-leads

andreyvelich avatar Apr 28 '24 19:04 andreyvelich

I think this update is much more confusing to new users and significantly confuses the message of "What is Kubeflow?".

To be clear, I am not against telling users that some components of Kubeflow can be installed on their own, but we need to be careful not to confuse people or make representations of support that are not true.


Alternate Proposal

The current page is actually pretty good at explaining the options users have to install Kubeflow.

I propose we simply add a 3rd option under "How to install Kubeflow?" so we have the following (in this order):

  1. Packaged Distributions
  2. Raw Manifests (advanced users)
  3. Standalone Components (advanced users)

We also must get firm commitments from the maintainers of components listed under "Standalone Components". Users will expect deployment support, and don't want to impose that on maintainers without their agreement.

thesuperzapper avatar Apr 29 '24 18:04 thesuperzapper

helpful when you want to try out locally or in your Kubernetes installation a specific component

@tarilabs I added your suggestion to the manifests installation, thanks! For the standalone installation, do you think these messages are not sufficient to explain value of Standalone Components ?

Kubeflow components can be deployed as standalone applications.
You can integrate those components to your existing AI/ML platform.
For example, for Model Training you can install Training Operator or for Model Serving you can install KServe.

andreyvelich avatar Apr 29 '24 20:04 andreyvelich

I think this update is much more confusing to new users and significantly confuses the message of "What is Kubeflow?".

@thesuperzapper Please can you explain why this update confuses users about Kubeflow Ecosystem value ? From my perspective we make it very clear that Kubeflow project can be used in various ways. The main motivation is to give new Kubeflow components (Model Registry, Spark Operator) visibility and grow the community around it (outside of Kubeflow Platform scope).

To be clear, I am not against telling users that some components of Kubeflow can be installed on their own, but we need to be careful not to confuse people or make representations of support that are not true.

Which message shows incorrect representation of support ? If users deploy Kubeflow components standalone and get some potential bugs/integrations issues, they should raise it to the components owners to get support.

Raw Manifests (advanced users)

What is the end-user value to tell that Raw Manifest and Standalone Installation is for advanced users ? Some Kubeflow components are very light-weight and easy to install.

Users will expect deployment support, and don't want to impose that on maintainers without their agreement.

I am happy to remove some of the Kubeflow components from this page if WG leads don't want their components to be presented here.

andreyvelich avatar Apr 29 '24 20:04 andreyvelich

cc @jeremyeder @bigsur0 Would love to get your feedback for this change as well.

andreyvelich avatar Apr 29 '24 20:04 andreyvelich

@andreyvelich

I have made an alternative PR which tries to be less confusing:

  • https://github.com/kubeflow/website/pull/3726

It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

thesuperzapper avatar Apr 29 '24 23:04 thesuperzapper

@andreyvelich I have made an alternative PR which tries to be less confusing:

* [Add "Standalone Components" section to "Installing Kubeflow" #3726](https://github.com/kubeflow/website/pull/3726)

It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

Doesnt it change this order here significantly?

- [**Install Kubeflow Components Standalone**](#install-kubeflow-components-standalone)
- [**Install Kubeflow Platform from Raw Manifests**](#install-kubeflow-platform-from-raw-manifests)
- [**Install Kubeflow Platform from Packaged Distributions**](#install-kubeflow-platform-from-packages-distributions)

seems to become

- [**Install Kubeflow Platform from Packaged Distributions**](#install-kubeflow-platform-from-packages-distributions)
- [**Install Kubeflow Components Standalone**](#install-kubeflow-components-standalone)
- [**Install Kubeflow Platform from Raw Manifests**](#install-kubeflow-platform-from-raw-manifests)

juliusvonkohout avatar Apr 30 '24 08:04 juliusvonkohout

@tarilabs I added your suggestion to the manifests installation, thanks! For the standalone installation, do you think these messages are not sufficient to explain value of Standalone Components ?

glad that helped, about the standalone very-imho is worthy to call out testing a component "locally" or in isolation, in addition to the existing AI/ML/K8s one might have. for example, as a relatively newcomer to Kubeflow, I thought I needed it all if I wanted to try one single part :) my2c, of course I can understand if this comment is not incorporated! 🚀 👍

tarilabs avatar Apr 30 '24 08:04 tarilabs

@andreyvelich I have made an alternative PR which tries to be less confusing:

It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

@thesuperzapper As we discussed, I am happy to incorporate your feedback in this PR based on our discussion during today's community call. A few thoughts from my side:


If you don't like the ordering of installation instructions we can make these 2 sections:

  • Install Kubeflow Components Standalone
  • Install Kubeflow Platform

In that case, we can move Distributions + Raw Manifests to the Install Kubeflow Platform section.


From my point of view, it doesn't give a lot of value for users to add Supports Standalone column to the table. We can always update the table with other Kubeflow Component, if it supports it. For example, we can add Kubeflow Notebooks in the future.


I am not sure if we should add the Note about limited or unavailable feature section to the Standalone Components. We can add the section to the Kubeflow Introduction:

  • What is Kubeflow ?
  • What is Kubeflow Standalone Components ?
  • What is Kubeflow Platform ?

And explain the differences there.

andreyvelich avatar Apr 30 '24 11:04 andreyvelich

Thanks everyone who participated in our today's discussion to show the real power of Kubeflow ecosystem. I strongly believe that this change will help us to grow the community especially with the new project adoption: Spark Operator and Model Registry.

I want to summarize our discussion today about this PR:

  1. We agree that it makes sense to explain that Kubeflow Components can be deployed standalone.
  2. We want to collect feedback from distribution owners for this change.
  3. We should re-use this ML Lifecycle diagram that @franciscojavierarceo and I created for Training Operator and re-use it in the ML Workflow guide
  4. We are discussing to add the following statement to the installation page: What is Kubeflow ? Example text:
Kubeflow is an ecosystem of various applications created to address each stage in the AI/ML lifecycle,
from exploration to training and serving. You can deploy Kubeflow components as standalone applications
or deploy Kubeflow as an end-to-end AI/ML platform. Anywhere you are running Kubernetes,
you should be able to run Kubeflow.

What is Kubeflow Standalone Components ?

TODO: Add text

What is Kubeflow Platform ?

TODO: Add text

If we agree with that, we can add these subsections to the installation page: Installing Kubeflow Standalone Components Installing Kubeflow Platform

In the future PRs we can split them between 2 separate pages if we want.

  1. If we agree, I can keep (Advanced User) and existing warnings for manifests installation in this PR. We can make followup discussion if we should change it.

  2. I am happy to take the table from PR: https://github.com/kubeflow/website/pull/3726, with these columns: Kubeflow Component, Source Code. As I said before: https://github.com/kubeflow/website/pull/3724#issuecomment-2085019170 I am not sure if we should have "Supported Standalone" section.

Please let me know if I missed something.

@thesuperzapper Please can you share recordings from our today's call.

andreyvelich avatar Apr 30 '24 21:04 andreyvelich

Distribution owners please give your feedback for this PR. /cc @ca-scribner @gkcalat @zijianjoy @Linchin @Tomcli @yhwang @johnugeorge @nagar-ajay @rimolive @thesuperzapper @liuqi @xujinheng @alexeadem

andreyvelich avatar Apr 30 '24 21:04 andreyvelich

@andreyvelich: GitHub didn't allow me to request PR reviews from the following users: Linchin, alexeadem.

Note that only kubeflow members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Distribution owners please give your feedback for this PR. /cc @ca-scribner @gkcalat @zijianjoy @Linchin @Tomcli @yhwang @johnugeorge @nagar-ajay @rimolive @thesuperzapper @liuqi @xujinheng @alexeadem

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-prow[bot] avatar Apr 30 '24 21:04 google-oss-prow[bot]

/cc @StefanoFioravanzo @akgraner for your feedback.

andreyvelich avatar Apr 30 '24 21:04 andreyvelich

/cc @chensun

gkcalat avatar May 01 '24 07:05 gkcalat

@andreyvelich thanks a lot for starting this. I had to drop early and didn't hear the discussion during the community call, but I think I have a pretty good context from the convo here. Based on your last comment, I can propose some text as basis for iteration:

Installing Kubeflow

What is Kubeflow?

Kubeflow is an open-source project designed to make deployments of machine learning (ML) workflows on Kubernetes simple, portable, and scalable. Its goal is to facilitate the orchestration of complicated machine learning pipelines and to empower users to deploy best-in-class open-source systems for ML to diverse infrastructures. Whether you’re a researcher, data scientist, ML engineer, or a team of developers, Kubeflow offers modular and scalable tools that cater to all aspects of ML workflows: from building models to deploying them to production.

[add diagram picture here]

What are Kubeflow Standalone Components?

Kubeflow is composed of multiple, independent components which address different aspects of a machine learning pipeline. These standalone components are designed to be usable both within the Kubeflow ecosystem and independently. For example [note: every component's name is a link to their respective installation page]

  • Katib for hyperparameter tuning.
  • Training Operator for distributed training and fine-tuning
  • Pipelines for workflow automation.
  • Model Registry for metadata and model lifecycle management

These components can be installed on their own on a Kubernetes cluster, providing flexibility to users who may not require the full capabilities of Kubeflow but wish to leverage specific functionalities.

What is Kubeflow Platform?

The Kubeflow Platform refers to the full suite of Kubeflow components bundled together with additional integration and management tools. Installing Kubeflow as a platform means deploying a comprehensive ML toolkit that integrates these components into a cohesive system, optimized for managing the full lifecycle of machine learning projects. This includes not only the standalone components but also:

  • Central dashboard for easy navigation and management.
  • Multi-user capabilities and access management.
  • Additional tooling and services for data management, visualization, and more.
  • Third-Party components, such as KServe [link] for scalable and production ready model serving

This integrated environment ensures that all the different pieces work together seamlessly, providing a more robust and streamlined user experience.

Deployment Methods

  1. Raw Manifests Deploying Kubeflow using raw manifests involves directly applying Kubernetes manifests to your cluster. This method offers maximum flexibility and control over the installation and configuration processes. It allows users to customize their setup extensively, adjusting Kubeflow to fit specific needs or constraints of their infrastructure. It's particularly suitable for users with specific requirements or those who prefer a hands-on approach to manage their Kubernetes resources. See [<link to subpage> Installing Kubeflow from Raw Manifests]
  2. Packaged Distributions Distributions are maintained by various organizations and typically aim to provide a simplified installation and management experience for Kubeflow. Distributions provide a more packaged approach, where Kubeflow comes pre-integrated with additional features or optimizations specific to certain cloud providers or technology vendors. These distributions might include enterprise-level support, additional security configurations, and seamless integration with other cloud services. This option is ideal for organizations looking for a more out-of-the-box solution that reduces the complexity of deployment and ongoing management, while still providing the robust capabilities of Kubeflow. See [<link to subpage> Kubeflow Packaged Distributions].

StefanoFioravanzo avatar May 03 '24 10:05 StefanoFioravanzo

Thank you for suggesting this @StefanoFioravanzo. From your perspective what is the best place to add those explanation ? E.g. we can add them to the https://www.kubeflow.org/docs/started/introduction/ or https://www.kubeflow.org/docs/started/installing-kubeflow/ pages.

I would modify it as follows:

What is Kubeflow ?

Kubeflow is a community and ecosystem of open-source projects to address each stage in the machine learning (ML) lifecycle. It makes ML on Kubernetes simple, portable, and scalable. Its goal is to facilitate the orchestration of Kubernetes workloads for ML and to empower users to deploy best-in-class open-source systems for ML to diverse cloud infrastructures. Whether you’re a researcher, data scientist, ML engineer, or a team of developers, Kubeflow offers modular and scalable tools that cater to all aspects of the ML lifecycle: from building ML models to deploying them to production for AI applications.

This statement addresses @jbottum points that Kubeflow is not only about code, but also about community of folks who know how to use Kubernetes for ML. I made this statement less ML Workflows focus, so users will get that Kubeflow Pipelines ≠ Kubeflow

In this statement, we can add the links to the ML Lifecycle from this PR: https://github.com/kubeflow/website/pull/3728

What are Kubeflow Standalone Components?

Kubeflow is composed of multiple, independent ~components~ open-source projects which address different aspects of a ~machine learning pipeline~ ML lifecycle. These standalone components are designed to be usable both within the Kubeflow ~ecosystem~ platform and independently. These components can be installed on their own on a Kubernetes cluster, providing flexibility to users who may not require the full capabilities of ~Kubeflow~ Kubeflow Platform but wish to leverage specific functionalities in the ML lifecycle.

(Do we need to link components here if we can link them in the installation page)?

What is Kubeflow Platform?

The Kubeflow Platform refers to the full suite of Kubeflow components bundled together with additional integration and management tools. Installing Kubeflow as a platform means deploying a comprehensive ML toolkit that integrates these components into a cohesive system, optimized for managing the ~full lifecycle of machine learning projects~ end-to-end ML lifecycle. ~This~ These includes not only the standalone components but also: Central Dashboard for easy navigation and management. Multi-user capabilities and access management. Additional tooling and services for data management, visualization, and more. ~Third-Party components, such as KServe [link] for scalable and production ready model serving~ This integrated environment ensures that all the different pieces work together seamlessly, providing a more robust and streamlined user experience.

I don't think we should tell our users that KServe is third-party component. Even that KServe is living in a separate GitHub org, it is still part of Kubeflow Core Components. We even list it as Kubeflow Components in the Architecture Diagram. I understand that we are still debating about it, but we have very special case just for KServe.

WDYT @StefanoFioravanzo ?

andreyvelich avatar May 03 '24 19:05 andreyvelich

@andreyvelich I thought we agreed to mostly leave the current "Installing Kubeflow" page, and create a NEW page specifically for "Installing Kubeflow Standalone Components"?

This helps users understand that there are two VERY distinct ways of using Kubeflow and keeps everyone happy, and reduces the amount of debate we need to have.

This would leave us with the following pages:

  • Installing Kubeflow Platform:
    • Renamed from the current "Installing Kubeflow" page
    • We can update the introduction to link to the other page.
    • Is only about using Kubeflow as a "platform" (with multi-user and authentication)
  • Installing Standalone Kubeflow Components:
    • A page which introduces the idea of using some components of Kubeflow on their own.
    • It should explain any differences in user experience from the "platform" versions of the components.
    • Uses a "table" to list the components like in: https://github.com/kubeflow/website/pull/3726/
    • Says that some distributions allow you to not deploy all components, without the standalone limitations.

Also, its critical that we don't misrepresent the relationship between KServe and Kubeflow, they are an independent project. Doing it right also opens up the possibility of having other independent projects that we treat in a similar way (e.g. Feast).

The "standalone components" page can just have two tables, like I did in my alternate PR (https://github.com/kubeflow/website/pull/3726):

Kubeflow Ecosystem:

Some components in the [Kubeflow Ecosystem](/docs/started/architecture/#conceptual-overview) may be deployed as standalone services, without the need to install the full platform.

{{% alert title="Note" color="dark" %}}
Some features may be __limited or unavailable__ when using standalone components.
For example, standalone Kubeflow Pipelines does not include multi-user support, and is limited to a single Namespace.

A few [packaged distributions](#packaged-distributions-of-kubeflow) allow you to choose which components are installed, while still providing the full version of those components.
{{% /alert %}}

The following table lists each component of Kubeflow and whether it supports standalone deployment:
Kubeflow Component Source Code Supports Standalone
Kubeflow Central Dashboard kubeflow/kubeflow No
Kubeflow Notebooks kubeflow/kubeflow No
Kubeflow Pipelines kubeflow/pipelines Yes | Installation Guide
Kubeflow Katib kubeflow/katib No
Kubeflow Training Operator kubeflow/training-operator Yes | Installation Guide
Kubeflow MPI Operator kubeflow/mpi-operator Yes | Installation Guide
Kubeflow Spark Operator kubeflow/spark-operator Yes | Installation Guide

External Add-ons:

There are a number of [External Add-Ons](/docs/external-add-ons/) which are commonly deployed alongside the Kubeflow ecosystem.
These tools are __maintained by external organizations__ and are not part of the core Kubeflow project.
However, some are included in the [Raw Kubeflow Manifests](#raw-kubeflow-manifests) (under the `contrib` folder), so are available in most [packaged distributions](#packaged-distributions-of-kubeflow).

The following table lists some popular add-ons and whether they are included in the raw manifests:
External Add-On Source Code In Kubeflow Manifests
KServe kserve/kserve Yes
Feast feast-dev/feast No

thesuperzapper avatar May 03 '24 20:05 thesuperzapper

I thought we agreed to mostly leave the current "Installing Kubeflow" page, and create a NEW page specifically for "Installing Kubeflow Standalone Components"? This helps users understand that there are two VERY distinct ways of using Kubeflow and keeps everyone happy, and reduces the amount of debate we need to have.

If we go with this approach the existing page will still require changes to explain that this installation is for Kubeflow Platform. It might be easier to start with single page with explanation of:

What is Kubeflow ?
What is Kubeflow Standalone Components
What is Kubeflow Platform ?

In the future we can split it, if we understand that single page with sub-sections confuse users. What do you think about it ?

It should explain any differences in user experience from the "platform" versions of the components.

Shouldn't it be WG and component responsibility to explain the differences ? I don't want user to feel overwhelmed in installation section. For example, we added some explanation to run Katib with Istio on Kubeflow Platform. Pipelines also have some docs in V1 section

Says that some distributions allow you to not deploy all components, without the standalone limitations.

We should discuss it with all active distributions to get their feedback. For example, we want to introduce conformance program in the future, and it will require distribution to support all Kubeflow Components even if users can select the desired components. We can include that statement to the Deployment Methods - Package Distributions in @StefanoFioravanzo suggestions.

Also, its critical that we don't misrepresent the relationship between KServe and Kubeflow, they are an independent project. Doing it right also opens up the possibility of having other independent projects that we treat in a similar way (e.g. Feast).

I have different thoughts here. KServe is graduated from KFServing which was part of Kubeflow Ecosystem. All other components don't have the same history. In the past 4 years, KServe community have provided ability to run KServe with Kubeflow Platform. I am happy to hear thoughts from other community members. cc @terrytangyuan @yuzisun @ckadner

The "standalone components" page can just have two tables, like I did in my alternate PR (https://github.com/kubeflow/website/pull/3726):

What is the reason to add supported standalone column to the table for users ? Why we can't just rely on @StefanoFioravanzo suggestion which explain what is included in the Kubeflow Platform (e.g. Central Dashboard) ?

andreyvelich avatar May 03 '24 21:05 andreyvelich

I think explaining what Raw Manifests and Package Distributions are is important as indicated in https://github.com/kubeflow/website/pull/3724#issuecomment-2092731855 by @StefanoFioravanzo

I also agree with https://github.com/kubeflow/website/pull/3724#issuecomment-2083591622. from @andreyvelich Raw Manifests are pretty easy to install and a quick way to get Kubeflow going. I think removing "advanced" is a good idea if we explain better what they are.

alexeadem avatar May 03 '24 23:05 alexeadem

Just adding some flavor here. The biggest feedback datapoint from Kubeflow Summit was that Kubeflow is hard to install. We don't have a Kubeadm for Kubeflow so the more we share with the world on how to "get started" the better. A good example is doing an install for a customer. I might say "let's get started with our initial install and we can work to tune the system based on your needs". That "tuning your system based on your needs" part makes sense for the standalone mode vs. platform mode we are discussing. The question I have here is how many users are focused on a more "DIY" KF taking specific components off the shelf vs. looking for an integrated solution with a plugin like experience to add features. "Good engineering is solving problems of today leaving room for the problems of tomorrow" kind of thing. I do know that for things like Kserve it seems comical to deploy an entire KF just to serve models, but if someone installs just KFP are they installing Kubeflow or just a component? I think we need to be very clear with the basic vs. advanced options and not give users decision fatigue. We ( I.E users that have been doing this for a hot minute) might be able to swap and switch components on the fly all day but we've got a lot of neural pathways we don't account for that might be painful for new users to walk :-). I also as always am a fan discussing the "what benefits are a standalone component vs. KF integrated component" including lifecycle and integration considerations. We might need to talk to users doing it both ways and figure out what drove them from one way to another. Could be a GREAT talk/knowledge dump for a summit too!

chasecadet avatar May 07 '24 14:05 chasecadet

I think it will be cleaner to have two separate "Installing kubeflow" pages:

  • Page 1: Installing Kubeflow Platform
  • Page 2: Installing Kubeflow Components

Key Reasons:

  1. We can make each page more focused and easier to follow.
  2. We can give more info about each component, without overwhelming users. For example, each component can get its own section heading on the "Installing Kubeflow Components" page. (See example)
  3. There are fundamentally two "user personas" for Kubeflow which align with these pages. Those looking for a full platform, and those only looking for one or more specific tools.

Example Structure:

  • Installing Kubeflow Platform:
    • What is Kubeflow Platform?
    • How to install the Kubeflow Platform?
    • Packaged Distributions of Kubeflow
    • Raw Kubeflow Manifests
  • Installing Kubeflow Components:
    • What are Kubeflow Components?
    • Standalone Components vs Kubeflow Platform
    • Component Index
      • (a table with links to the following sections)
      • (columns: Component, Purpose, Docs, Standalone Compatible)
    • Kubeflow Components
      • Kubeflow Central Dashboard
      • Kubeflow Notebooks
      • Kubeflow Pipelines
      • Kubeflow Katib
      • Kubeflow Training Operator
      • Kubeflow MPI Operator
      • Kubeflow Spark Operator
      • Kubeflow Model Registry
    • External Add-Ons:
      • KServe
      • Feast

Under each component-specific heading, we can explain very briefly what that component is, and and differences between using that component standalone compared with as part of a platform.

thesuperzapper avatar May 07 '24 18:05 thesuperzapper

@thesuperzapper Why we don't want to separate installation between 2 pages in the future ? We can start with single page, and if users will get confused, we will split ?

We can give more info about each component, without overwhelming users. For example, each component can get its own section heading on the "Installing Kubeflow Components" page. (See example). Under each component-specific heading, we can explain very briefly what that component is, and and differences between using that component standalone compared with as part of a platform.

Why do we need it if that can be part of components installation section? Kubeflow Pipelines can add that notice in their installation section. @StefanoFioravanzo was working to refactor components docs to follow the same structure as we did for Katib:

There are fundamentally two "user personas" for Kubeflow which align with these pages. Those looking for a full platform, and those only looking for one or more specific tools.

I think, that is not always true as we discussed on the last call. Some users might start to explore Kubeflow Ecosystem from distributions, some of them might start explore Kubeflow Ecosystem from Standalone Components. And they can choose whatever give them more value.

andreyvelich avatar May 08 '24 12:05 andreyvelich

@StefanoFioravanzo I added section based on our discussion above: https://github.com/kubeflow/website/pull/3724#issuecomment-2092731855 and https://github.com/kubeflow/website/pull/3724#issuecomment-2093677592. Please let me know how does it sound to you.

andreyvelich avatar May 08 '24 12:05 andreyvelich

@thesuperzapper I like your approach a lot. You've thought this through, and it shows. One thing that I think would be interesting to investigate, and maybe even a blocker here, is conformance. I am out of the loop, and maybe @terrytangyuan can guide me, but will conformance be on the individual components or the entire "Kubeflow solution". This is particularly interesting when we start socializing and talking about Kubeflow to a broader audience. At what point is something using multiple components, "Kubeflow," vs. an ML solution that happens to use Kubeflow components where necessary, like what DeployKF seems to be going for? This becomes interesting when starting to discuss distributions. Is Kubeflow just a community supporting a bunch of indvidual components? A platform? How do we define something as a "part of Kubeflow" and certify it? The first page, "installing the Kubeflow platform," means we must define how many or few of the components make a "Kubeflow platform"? How do we talk about this? What are the boundaries of the system? I would love to learn more!

chasecadet avatar May 09 '24 01:05 chasecadet

After discussing this with @kubeflow/kubeflow-steering-committee, we proposed the following changes:

  • Move to the introduction:
    What is Kubeflow?
    What is Kubeflow Standalone Components?
    What is Kubeflow Platform?
    
  • Order the installation page as follows: standalone components -> packaged distributions -> raw manifests. We agreed that this is the simplest way for new users to get started with Kubeflow ecosystem and don't get blocked during the installation phase to see the value of Kubeflow ecosystem.
  • Update the standalone components table with column which indicates which stage of ML lifecycle it solves.
    • I made the order similar to ML lifecycle: Data Preparation -> Model Development -> Model Optimization -> Model Training -> Model Registry -> Model Serving. What do you think about this order ?
  • We want to keep the KServe in this table to indicate that this component will always solve the Model Serving stage in the Kubeflow ecosystem. cc @yuzisun

andreyvelich avatar May 10 '24 22:05 andreyvelich

@andreyvelich Please link directly to https://github.com/kubeflow/manifests?tab=readme-ov-file for the manifests WG.

Please also remove the link to https://github.com/kubeflow/community/tree/master/wg-manifests which is heavily outdated and misleading

So

Install Kubeflow Platform from Raw Manifests

The raw Kubeflow Manifests are aggregated by the [Manifests Working Group](https://github.com/kubeflow/manifests?tab=readme-ov-file) and are intended to be used as the base of packaged distributions and for advanced users.

Kubeflow Manifests contain all Kubeflow Components, Kubeflow Central Dashboard, and other Kubeflow applications which makes Kubeflow Platform. This installation is helpful when you want to try out the end-to-end Kubeflow Platform capabilities.

Users may choose to install the manifests for a specific Kubeflow version by following the instructions in the README of the [kubeflow/manifests](https://github.com/kubeflow/manifests?tab=readme-ov-file) repository.

juliusvonkohout avatar May 13 '24 06:05 juliusvonkohout

/lgtm

terrytangyuan avatar May 13 '24 13:05 terrytangyuan

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: terrytangyuan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow[bot] avatar May 13 '24 13:05 google-oss-prow[bot]

ubeflow Manifests contain all Kubeflow Components, Kubeflow Central Dashboard, and other Kubeflow applications which makes Kubeflow Platform. This installation is helpful when you want to try out the end-to-end Kubeflow Platform capabilities.

@juliusvonkohout I removed the Manifests WG link. @kubeflow/wg-manifests-leads Please can you update the charter to keep it up-to-date ?

This installation is helpful when you want to try out the end-to-end Kubeflow Platform capabilities.

We do have redirect to the installation README below:

Following the instructions in the README of the kubeflow/manifests repository.

andreyvelich avatar May 13 '24 13:05 andreyvelich

/lgtm

diegolovison avatar May 13 '24 16:05 diegolovison

ubeflow Manifests contain all Kubeflow Components, Kubeflow Central Dashboard, and other Kubeflow applications which makes Kubeflow Platform. This installation is helpful when you want to try out the end-to-end Kubeflow Platform capabilities.

@juliusvonkohout I removed the Manifests WG link. @kubeflow/wg-manifests-leads Please can you update the charter to keep it up-to-date ?

This installation is helpful when you want to try out the end-to-end Kubeflow Platform capabilities.

We do have redirect to the installation README below:

Following the instructions in the README of the kubeflow/manifests repository.

@andreyvelich here is the update PR https://github.com/kubeflow/community/pull/720

juliusvonkohout avatar May 14 '24 08:05 juliusvonkohout