kubeflow
Fine-Tune APIs for LLM Documentation
This issue tracks the Kubeflow 1.9 documentation deliverables for the new Fine-Tune APIs for LLMs.
-
[ ] Write intro doc with high-level presentation and user stories. @StefanoFioravanzo Draft to be turned into a docs webpage https://docs.google.com/document/d/18PuuaDRISj5mlrBn1GJrxwuB6Z5zTtXKpVbLUIeLx-8/edit?usp=sharing Architecture doc. @andreyvelich API docs
Everything tracked here https://github.com/kubeflow/website/pull/3718
-
[ ] Tutorial
cc @kubeflow/wg-training-leads @deepanker13 cc release team docs leads @diegolovison @hbelmiro
This feature is based on this work:
- https://github.com/kubeflow/training-operator/pull/1659
do we have supporting documentation for TFJob and PyTorchJob Function APIs, should we track it separately of include it in the architecture/API doc above?
@StefanoFioravanzo I can help with the tutorial. Also do you have any reference for api documentation?
Example Notebooks can be found here:
- https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/text-classification/Fine-Tune-BERT-LLM.ipynb
- https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/language-modeling/train_api_hf_dataset.ipynb
@andreyvelich @deepanker13 are we writing a tutorial based on these APIs eventually?
@andreyvelich @deepanker13 are we writing a tutorial based on these APIs eventually?
We already have this Notebook to try out this feature: https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/text-classification/Fine-Tune-BERT-LLM.ipynb I think, initially we can just link this Notebook in the website.
Ok! Where do you suggest we link it?
I already linked it here: https://deploy-preview-3718--competent-brattain-de2d6d.netlify.app/docs/components/training/user-guides/fine-tuning/#next-steps
