kubeflow
Data WG roadmap for KF 1.10
Hello @kubeflow/wg-data-leads @tarilabs @ChenYi015
The 1.10 release team will use this tracking issue to coordinate the integration of the Model Registry and Spark Operator with the Kubeflow 1.10 release.
Heads up that the feature freeze is scheduled for January 20th 2025 (needs confirmation). For the full release schedule, refer to the 1.10 release timeline.
(note: links are pending due to https://github.com/kubeflow/community/pull/761)
To prepare for the 1.10 release, please provide answers to the following questions:
- What version of Model Registry and Spark Operator would you like to include for the 1.10 release?
- Could you provide a branch/tag for this version? (note: It doesn't have to be final)
- Can you share a link to all open / work in progress issues that you're expecting to include in the 1.10?
- If you're using the following dependencies, could you provide the versions you'll be supporting for each dependency?
- KServe/KNative
- Istio
- Kubernetes
- Cert-Manager
- Kustomize
The release team WG liaison for Pipelines WG will be @milosjava and you can keep track of high-level release updates in the https://github.com/kubeflow/community/issues/768 issue.
from Model Registry WG
- theme:
Kubeflow Model Registry UI - roadmap/details: https://github.com/kubeflow/model-registry/issues/175
cc/ @milosjava @rareddy @ederign
From Spark Operator:
- roadmap: https://github.com/kubeflow/spark-operator/issues/2193
We need a maintainer for the spark manifests at https://github.com/kubeflow/manifests/tree/master/contrib/spark. See also https://github.com/kubeflow/manifests/blob/master/proposals/20220926-contrib-component-guidelines.md#component-requirements for PSS restricted requirements and there should be a synchronization script in /hack even if it just resembles the current makefile
We need a maintainer for the spark manifests at https://github.com/kubeflow/manifests/tree/master/contrib/spark. See also https://github.com/kubeflow/manifests/blob/master/proposals/20220926-contrib-component-guidelines.md#component-requirements for PSS restricted requirements and there should be a synchronization script in /hack even if it just resembles the current makefile
@milosjava to follow up on this
Model registry , no red flags , blockers…. Progressing well on the development. What is still pending to be 'ready' is the integration with Kubeflow Central Dashboard and also how we are doing to deploy our components in the manifests repo... but this is already in progress
Model Registry no blockers, we can say that approx 50% is done. As Josh suggested on a community call we will find an end-user and go through docs together. My idea for the end user is one of our (pepsico) projects.
Regarding SparkOperator and the follow up:
- Might need to create a Kustomize manifest for the operator sincemost users and the dev work that was done has been on the Helm chart
- The doc work can be done
- The existing end to end tests should be fine for the testing but one could be pulled out as a specific smoke test
- Some work on the security stance is done but need to follow up with Julius for some other changes
To summarize: Smoke test, security stance, Kustomize and doc work are doable. Need more info on Integration between the operator and Kubeflow Here is the issue to track If that is fixed we can include it in Kubeflow 1.10
Also I think would be possible to find end-user for SparkOperator as well.
@milosjava
Spark-operstor is already in our CICD. Here are the remaining issues to promote it to an application https://github.com/kubeflow/manifests/issues/2912
@rimolive Sharing model registry progress track: https://github.com/kubeflow/model-registry/milestones
UI road map : https://github.com/kubeflow/model-registry/issues/175#issuecomment-2498520482
from Milos in Release meeting:
- Model Registry tracker and fine-grainer tracker for UI per https://github.com/kubeflow/community/issues/769#issuecomment-2498535490
- spark operator is OK but there is some integration work ongoing for Kubeflow
Model Registry, still in green, no blockers. current focus on https://github.com/kubeflow/model-registry/pull/599
@milosjava mentions in the Release meeting:
- MR progressing there is a call following the Release meeting
- Spark operator did not respond to request for feedback/update (monthly call is a bit TZ difficult)
Model Registry progressing, no issues.
Spark Operator: seems there is an issue with integration with Kubeflow. I am in the contact with number of Spark Operator developers and it seems istio support is problem and probably we need help with this. More info: https://github.com/kubeflow/manifests/issues/2912#issuecomment-2544831739
@milosjava can you provide an update on the release? please let us know if there's any blockers/issues Feature freeze is Jan 20th.
@varodrig Update is that we are in Green. There was not a lot of activity over the holidays but there are no blockers nor red flags and it seems we will be ready for the feature freeze.
@varodrig @rimolive MR and SparkOperator no blockers or red flags. Spark Operator docs -> https://github.com/kubeflow/website/tree/master/content/en/docs/components/spark-operator
Model Registry biweekly 2024-01-20 confirmed the Dashboard integration to MR UI code development was done, but what is missing is proper access control settings, to make it work ootb following the tutorial. The ui team will reach out to Platform/Manifest WG for making sure the solution of designating a specific namespace (ie kubeflow-model-registries) does not have any further drawback (it does not have concerns from MR WG). The tutorial will be updated to reflect. Further discussion on adjacent topics of dashboard (ie its manifest) have been discussed and summarized here: https://github.com/kubeflow/model-registry/issues/705#issuecomment-2603100651 @milosjava @varodrig @rimolive @ederign
Model Registry biweekly 2024-01-20 confirmed the Dashboard integration to MR UI code development was done, but what is missing is proper access control settings, to make it work ootb following the tutorial. The ui team will reach out to Platform/Manifest WG for making sure the solution of designating a specific namespace (ie
kubeflow-model-registries) does not have any further drawback (it does not have concerns from MR WG). The tutorial will be updated to reflect. Further discussion on adjacent topics of dashboard (ie its manifest) have been discussed and summarized here: kubeflow/model-registry#705 (comment) @milosjava @varodrig @rimolive @ederign
thank you @tarilabs for the information. cc @tarekabouzeid @juliusvonkohout