kubeflow
website: Empowering Kubeflow Documentation with LLMs
What would you like to be added
In our previous Kubeflow Community Calls, we discussed whether we can leverage the power of LLMs to improve Kubeflow Documentation.
I think, this could be a great project for Kubeflow GSoC 2025 if we can find mentors to drive it!
Let's use this issue to track it.
Community Meeting recordings: https://youtu.be/Ejvw2FjqqMI?t=2044 Meeting Notes:
- We can start with some simple prompts (e.g. where to find Kubeflow Trainer repo?)
- vLLM uses RunLLM for their docs: https://runllm.com/
/area gsoc
cc @shravan-achar
Love this idea? Give it a 👍.
Also what exactly do you expect from this what I have in mind is like an assistive chatbot that can give us links and answer our questions. So when we ask questions like hey where is the repo it'll respond with hey its available on github or open this link.
So basically a bot that talks to the docs knowledge base and also provides us with links.
Can anyone give a reply or a response so I can start working on a POC. As this was noted in a meeting which was done over 2 months ago....
Hii is the idea here to build a chatbot from the documentation and then create new documentation using a GPT family model?
@SanthoshToorpu @staru09 are either of you looking to do this for GSOC?
Adding website link for more information https://www.kubeflow.org/events/gsoc-2025/
I’m interested in working on this as my GSoC project! I was wondering—have we considered using RAG (Retrieval-Augmented Generation) for the LLM? It could enhance accuracy by dynamically retrieving relevant documentation.
I came across an interesting Hugging Face article on how they use RAG to improve documentation search. Here’s the link: Advanced RAG on Hugging Face Docs
By the way, what do we want to get out of the LLM for docs? Are we aiming for a better search feature?
If that’s the case, I noticed that Issue #4029 discusses a similar feature but seems to take a simpler approach. Would it make sense to align these ideas, or should we explore them as separate solutions?
Looking forward to your thoughts!
@SanthoshToorpu @staru09 are either of you looking to do this for GSOC?
Depends if you guys support and give active feedback I may close it before Gsoc and search for something new for gsoc contribution. But this needs iterative development. I've got good amount of experience with RAG n considering the simple documentation we can wrap this up in a few days. So it's your call...
@SanthoshToorpu @staru09 are either of you looking to do this for GSOC?
I am looking forward to do this as a summer project and otherwise as well. I can start working on POC based on my project that I did in my LFX mentorship at wasmedge.. I am thinking of adding advanced features like evals for accurate responses, test time scaling(chain of thought) and LLM as judge to generate the most accurate result for a query. This is something I am working on my project with UC Berkeley. So I think these features can be implemented in this project and it'll be good summer project to work on. Is there something that you guys can recommend so I can start working on it.
Thanks Aru
@SanthoshToorpu @staru09 Re:GSOC
If you contribute directly to this issue you might not be eligible for summer of code (which I believe they pay you for)
To participate in GSoC with Kubeflow, you must meet the GSoC eligibility requirements:
-
Be at least 18 years old at time of registration. -
Be a student or an [open source beginner](https://developers.google.com/open-source/gsoc/faq#how_do_i_know_if_i_am_considered_a_beginner_in_open_source_development). -
Be eligible to work in their country of residence during duration of program. -
Be a resident of a country not currently embargoed by the United States.
Steps to Apply for GSoC with Kubeflow
1. Sign Up as a Student
- Register on the GSoC website.
2. Join the Kubeflow Community
- Join the Kubeflow Slack.
NOTE: Please do not reach out privately to mentors. Instead, start a thread in the #kubeflow-contributors channel so others can see the response.
3. Learn About Kubeflow
- Read the Introduction to Kubeflow.
- Review the Architecture Overview.
- Consider trying out Kubeflow (not required, can be challenging).
4. Review the Project Ideas
- Decide which projects interest you.
- You may wish to attend the next community meeting for the group leading your chosen project.
NOTE: While we recommend submitting a proposal based on the project ideas, you can also submit a proposal with your own idea.
5. Submit Your Proposal
- Submit a proposal through the GSoC website between March 24th and April 8th.
6. Wait for Results
- GSoC will announce results on May 8th.
Otherwise, if you are interested in contributing to this as an individual that would be great too.
Let's then develop a proposal and discuss. Maybe if a GSOC users I interested we can help them engage.
I'd expand this proposal to use VLLM Kserve integration, demonstrate ingestion with KFP, determine how to measure good response, determine a way to get response feedback, and optional use of Katib to enhance document retrieval, temperature (cc @varshaprasad96 ). If one wanted to go above and beyond, we could implement https://docs.feast.dev/reference/alpha-vector-database. (@franciscojavierarceo could provide feedback/support on this).We also want to ensure the responses are timely and reference the articles they are pulling from if a user wants "lineage" or to dive deeper into a primary source. Thus, we can fact check ourselves and improve documentation.
Take on what you see fit. We will need to come up with a plan, determine services/architecture, and go from there.
Okay let's expand the proposal as this will be a very trivial task for anyone with basic knowledge lets discuss more in this thread and slack and comeup with a better proposal. With that lets make this GSOC worth the wait...
I think it makes more sense to draft a proposal and discuss this offline via slack.
Okay let's expand the proposal as this will be a very trivial task for anyone with basic knowledge lets discuss more in this thread and slack and comeup with a better proposal. With that lets make this GSOC worth the wait...
One thing I want to make certain, is the goal is to use Kubeflow and Kubeflow components to do this with a pipeline for updates to docs, embedding etc.. to show e2e Rag with Kubeflow. This is a "dog fooding exercise" If you want to take ownership and believe this to be trivial, please let us know and we can remove it from GSOC. All I ask is you take point on it.
Well theres difference in between getting things done and demonstrating that kubeflow can get things done. If we are doing this as an e2e tutorial that we can do stuff like making rag in kubeflow and actually make a tutorial outof it then it is good enough for gsoc. As there is no limit for tutorials and we can extend the boundaries as far as we want. But if this is just adding a feature to the kubeflow website, I wouldn't recommend this for GSOC. As we have a great community people who can do great things either we need to expand the proposal or think something else.
Or instead of just getting documentation we can make a proper RAG application made using kubeflow for kubeflow. Where instead of simply giving docs snippets or help stuff it gives code and steps to actually get things done. Now that is something worth GSOC as we are packaging this awesome tech into a single interface..
@SanthoshToorpu you seem very passionate about this topic and have a lot of experience in this space! I am going to lean on your wisdom here as an SME. My asks:
- Would you be willing to scope out level of effort to add this to the website?
- What details would you need to know?
- What infrastructure is necessary?
- What is the proposed flow?
- The stack?
- What Model?
- Do you foresee us needing to fine tune?
- Would you use the fine-tuning operator?
- What would a hand off look like?
- What will the level of effort to continue supporting this feature? Then we can ensure we get you the proper resources and details.
@andreyvelich do we have a typical format for all the above KEP style?
Okay let's expand the proposal as this will be a very trivial task for anyone with basic knowledge lets discuss more in this thread and slack and comeup with a better proposal. With that lets make this GSOC worth the wait...
One thing I want to make certain, is the goal is to use Kubeflow and Kubeflow components to do this with a pipeline for updates to docs, embedding etc.. to show e2e Rag with Kubeflow. This is a "dog fooding exercise" If you want to take ownership and believe this to be trivial, please let us know and we can remove it from GSOC. All I ask is you take point on it.
The available ecosystem kubeflow ecosytem will be the deciding factor for this project. I mean current ecosystem is mature enough to create a RAG based chatbot but for a summer project something advanced and meaningful can be proposed. One of the key area is to control hallucinations by the model and check the accuracy of the responses. Another area is to see the cost of operation for this bot. One area could be the audience it'll be serving i.e. first time users trying the kubeflow or some used to developers who will be more interested in generating some useful code from the bot. So maybe we can create a mixture of agents and then explore finetuning or RAG based methods as both have their pros and cons.
So I have some opinions on how we could structure this with RAG, but I'd like candidates to draft proposals instead using the Kubeflow components (this would incldue Feast for RAG).
Implementing a chatbot would be one way to empower the KF Documentation with LLMs, but it is technically not the only way, so I think there's a lot of opportunity for candidates to create novel proposals. Of course we could be prescriptive here but I think there's some fun in the creativity.
@franciscojavierarceo wondering if have ideas for the outcome and scope? What's "good" look like and what do you see as the end outcome of this. I think that will give folks a domain in which to be creative within. A "North Star" so to speak, especially since the goal is to be in service of this issue and proposal.
Yeah, that sounds good. I'll provide some thoughts tomorrow. 👍
Yeah, that sounds good. I'll provide some thoughts tomorrow.
Awesome! We could be seeing scope creep or shrinkage so it will be good for us all to discuss what this looks like when "complete" especially in reference to the GSOC. I do love leaving it a bit open ended so contributors can determine what a good first pass ready for iteration looks like and we can then rope in more folks to contribute once we've got a good foundation. Great discussion!
Adding some thoughts:
For a GSoC project, wherein we have limited time but would like to be able to get a useful outcome out of it towards the end that provides a good experience for the student, and benefits the community, we can probably guide the applicants to pick one/more of the outcomes below:
- Creating a chatbot with better search relevance, for eg: chatbot should be able to retrieve accurate documentation links based on user queries (similar to what @chasecadet mentioned earlier). This could involve:
- Better indexing of documentation content.
- Implementing embeddings-based search (e.g., using OpenAI, Cohere, or Elasticsearch).
- Automated Responses for Common Issues like Identify FAQs from GitHub issues and Slack discussions on Kubeflow, Train the chatbot to suggest solutions from existing user comments, rank and point them.
-
Develop an interactive Q&A workflow where the chatbot helps users debug installation issues by asking relevant questions (e.g., “Which platform are you deploying on and what's your use case?” → Suggests the right guide, and steps).
-
For long term maintenance - Integration with CI/CD for Documentation Updates:
- Ensure the chatbot pulls the latest documentation whenever updates are made (a webhook to trigger when a PR with a feature merges?).
- Automate alerts when a referenced page is missing or outdated.
Students can probably be asked to submit a proposal detailing how they would develop these features using Kubeflow components. This will allow the mentors to evaluate their approach and then we can guide them toward the best implementation strategy.
@SanthoshToorpu you seem very passionate about this topic and have a lot of experience in this space! I am going to lean on your wisdom here as an SME. My asks:
- Would you be willing to scope out level of effort to add this to the website?
- What details would you need to know?
- What infrastructure is necessary?
- What is the proposed flow?
- The stack?
- What Model?
- Do you foresee us needing to fine tune?
- Would you use the fine-tuning operator?
- What would a hand off look like?
- What will the level of effort to continue supporting this feature? Then we can ensure we get you the proper resources and details.
@andreyvelich do we have a typical format for all the above KEP style?
Hey I'm just a learner and enthusiast but I'll surely be helping.
Yes This can be added to website. Probably Nothing but I need to know how to handle the feedbacks Like what exactly yall use something like JIRA or GH issue. Currently for the testing honestly nothing is required. The modern LLM stack is just Full stack dev. But A lil knowledge and Cot's would make this even better. Proposed flow :
- Gather the data.
- Test different chunksizes and overlaps in FAISS(Local setting.)
- Test a few light weight models.
- Use a proper commercial model( If there is financial Backing) Stack is pretty generic build using python and inject it into an empty div tag in the exisiting code base. Model any llama does well no need of reasoning models until we plan proper code generation. No need of finetuning as current models have a huge context window. RAG would just be fine. If interested we can start with Lora adapter from the peft library and try few variations. Hand off part simple git code push would do well. But for code injection we need good amount of testing. Yes I'll be happy to contribute till a couple of years n more (Love the community though)
Anything please lemme know
That is the prod part but if we want to make a E2E tutorial that yes Kubeflow is transforming RAG then this will be a good lvl thing.
Doing yes anyone can do this but as @chasecadet suggested lets make this a kubeflow tutorial a proper one which will allow any user not project dudes but sass comapnies can use this tutorial to add a propr RAG with flexibility to their sites. I worked in a couple of Saas.
More discussion on this in Slack.
More discussion on this in Slack
You can use the Kubeflow-contributors slack if you want to start a thread, or you can 1000% continue to comment here. I believe everyone on this thread wants to help with an awesome initiative. Whether it's accepted as a GSOC project or simply a rag tag band of OSS contributors, I know we are going to build something awesome and help show "rag in the real world".
Created a doc with some ideas for GSOC folks on what this could look like. There are lots of layers you can work on and tune to get this project started. I recommend not biting off more than you can chew and focus on a good first iteration or really tuning one aspect as far as you can so others can layer their contributions on top of it. The world is your oyster!
@SanthoshToorpu @varshaprasad96 @staru09 @Dylannni.
also @franciscojavierarceo feel free to make suggestions and add anything that Feast might help with or how the folks can think about Feast in their future proposed stacks. Thanks everyone!
Created a doc with some ideas for GSOC folks on what this could look like. There are lots of layers you can work on and tune to get this project started. I recommend not biting off more than you can chew and focus on a good first iteration or really tuning one aspect as far as you can so others can layer their contributions on top of it. The world is your oyster!
@SanthoshToorpu @varshaprasad96 @staru09 @Dylannni.
also @franciscojavierarceo feel free to make suggestions and add anything that Feast might help with or how the folks can think about Feast in their future proposed stacks. Thanks everyone!
Neat... I've got a few addons. I'll add my comments 🙌
From the doc
Pipeline Abstractions and Automation Using Kubeflow Pipelines for RAG Workflows Managing pipeline workflows in Kubeflow Pipelines to streamline document processing Implementing parameterized pipeline components to allow for flexible ingestion and retrieval tuning. Ensuring modular execution to adapt to changes in Kubeflow documentation without requiring a full reprocessing.
- @chasecadet can you give here examples/docs of using Kubeflow Pipelines for similar tasks.
- For data ingestion, what type of synchronization is expected? For example, one way could be to add a scheduled check for updates on docs. Other could be a workflow. Any preferences on this.
- Whats the expected frontend for this chatbot. For example, there could be a chatgpt like webapp, or a slackbot, or an User facing API so that it can be further integrated in other places, or a combination of these.
- which slack channel is used for discussion on this?
@andreyvelich for those that are interested in using RAG, can we add to the documentation that Feast is the recommended Kubeflow component? I've gotten quite a number of inquiries about it. 😅
This is Anshuman Awasthi, a Masters student at Indian Institute of Information Technology Lucknow, India. This project sounds fun but highly challenging. Would like to take this up.
@SanthoshToorpu Hi, with all due respect to your approach i think the goal is to leverage LLMs to improve Kubeflow documentation we can go with a more simpler and faster approach because RAG-Based FAISS Approach is used if the we have large, unstructured documentation and need semantic search for better understanding. Since Kubeflow docs are structured, a simpler and faster approach i have while still giving LLMs the context they need. @franciscojavierarceo @andreyvelich
@SanthoshToorpu Hi, with all due respect to your approach i think the goal is to leverage LLMs to improve Kubeflow documentation we can go with a more simpler and faster approach because RAG-Based FAISS Approach is used if the we have large, unstructured documentation and need semantic search for better understanding. Since Kubeflow docs are structured, a simpler and faster approach i have while still giving LLMs the context they need. @franciscojavierarceo @andreyvelich
You abstracted it way too much the point is if we want certain issues we have to refer repo and issue threads also specific code snippets and certain usecase based tutorials are not available directly on the website. We need component based repo knowledge for that. Yes your point is absolutely correct if we are planning to just make a docs answering bot. But we need something more than that. Reading repo's and contributing to them is the essence of open source and that essence is captured in the right way with RAG.