airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Add Supervised Fine Tuning Train Operator, Hook, Tests, Docs

Open CYarros10 opened this issue 1 year ago • 0 comments

This pull request adds the following:

SupervisedFineTuningHook: Hook for Google Cloud Vertex AI Supervised Fine Tuning APIs. SupervisedFineTuningTrainOperator: Use the Google Cloud Supervised Fine Tuning API to create a tuning job.

About Model tuning: a crucial process in adapting Gemini to perform specific tasks with greater precision and accuracy. Model tuning works by providing a model with a training dataset that contains a set of examples of specific downstream tasks.

A sample DAG containing these operators could look like: JSONL training data arrives in GCS >> GCSObjectExistenceSensor >> SupervisedFineTuningTrainOperator >> GenerativeModelGenerateContentOperator

CYarros10 avatar Aug 27 '24 20:08 CYarros10