airflow
airflow copied to clipboard
Add Supervised Fine Tuning Train Operator, Hook, Tests, Docs
This pull request adds the following:
SupervisedFineTuningHook: Hook for Google Cloud Vertex AI Supervised Fine Tuning APIs. SupervisedFineTuningTrainOperator: Use the Google Cloud Supervised Fine Tuning API to create a tuning job.
About Model tuning: a crucial process in adapting Gemini to perform specific tasks with greater precision and accuracy. Model tuning works by providing a model with a training dataset that contains a set of examples of specific downstream tasks.
A sample DAG containing these operators could look like: JSONL training data arrives in GCS >> GCSObjectExistenceSensor >> SupervisedFineTuningTrainOperator >> GenerativeModelGenerateContentOperator