oss-fuzz-gen icon indicating copy to clipboard operation
oss-fuzz-gen copied to clipboard

Dynamically assert model temperature value in argparser

Open dberardi99 opened this issue 9 months ago • 3 comments

A new method get_model_temperature(args: argparse.Namespace) has been added to hardcode the model temperature value based on the model name. The chat session models (namely the ones ending with "-chat" or "-azure") have been treated as the corresponding base models. The temperature values can be found in the following data table https://github.com/google/oss-fuzz-gen/issues/366#issuecomment-2708184530

The temperature value will be automatically aligned to the one of the model chosen only if no temperature has been set (args.temperature == TEMPERATURE). Instead, if a wrong model name is fed (args.model in models.LLM.all_llm_names() == 0), the above method is skipped and the temperature is left unchanged to its default value.

In addition, the default temperature has been changed to 1.0 since it is the one relative to the default model (vertex_ai_gemini-1-5)

Fix https://github.com/google/oss-fuzz-gen/issues/366

dberardi99 avatar Mar 12 '25 04:03 dberardi99

Hi @DonggeLiu Could you please review it when you get a chance? Many thanks in advance!

dberardi99 avatar Mar 14 '25 00:03 dberardi99

@DonggeLiu I'm so sorry for the great delay... If you like this final implementation I can open a new PR creating a new branch for this fix in order to leave the modification history clearer.

dberardi99 avatar Apr 12 '25 17:04 dberardi99

Thanks @dberardi99. This is already pretty good, just a final question regarding reliability.

I don't think you have to create a new PR, as the modification history is pretty clear. But please let me know if you prefer otherwise.

Also, as your code is almost ready to merge, I will create a new base (#986) for your PR so that we can experiment with it before merging into main.

Awesome! We can continue working in this PR with no problem in this case

dberardi99 avatar Apr 14 '25 06:04 dberardi99