generative-ai icon indicating copy to clipboard operation
generative-ai copied to clipboard

[Bug]: Tuning Gemini with 500 rows takes endless hours and never stops

Open sidoncloud opened this issue 1 year ago • 1 comments

File Name

https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/tuning/gemini_supervised_tuning_qa.ipynb

What happened?

Tuning Gemini with 500 rows takes and basic minimum hyperparam values takes 3+ hours if you cancel the job after 3 hours else it goes on and on forever and you'll see a hefty cloud bill - This is something all google cloud paying users need to be made aware of before they try out something that does not work.

Relevant log output

No output

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

sidoncloud avatar May 04 '24 13:05 sidoncloud

I launched two different tunings, with a 660MB and 1100MB dataset, 18 hours ago. The first one is now 5% processed, and the second one is still validating the dataset. Now, the Gemini tuning is in preview, and it doesn't cost anything, but I am unsure if I would do it if it was billed. It's heartbreaking how long it takes.

Ali-Razmjoo avatar Jun 18 '24 09:06 Ali-Razmjoo