gorilla icon indicating copy to clipboard operation
gorilla copied to clipboard

[BFCL] Add gemini-1.5-pro-002, gemini-1.5-pro-002-FC, gemini-1.5-pro-001, gemini-1.5-pro-001-FC, gemini-1.5-flash-002, gemini-1.5-flash-002-FC, gemini-1.0-pro-002, gemini-1.0-pro-002-FC

Open HuanzhiMao opened this issue 1 year ago • 2 comments

This PR adds the following new models to the leaderboard:

  • gemini-1.5-pro-002
  • gemini-1.5-pro-002-FC
  • gemini-1.5-pro-001
  • gemini-1.5-pro-001-FC
  • gemini-1.5-flash-002
  • gemini-1.5-flash-002-FC
  • gemini-1.0-pro-002
  • gemini-1.0-pro-002-FC

Note: All the code for Gemini prompting series inference are already written out in #644, but they were not thoroughly verified at that time due to limited bandwidth. This PR has the code tested and thus we 'officially' add support for the prompting version of the Gemini models.

HuanzhiMao avatar Sep 24 '24 10:09 HuanzhiMao

Update: Add gemini-1.5-pro-002 and gemini-1.5-pro-002_FC in this PR as well.

HuanzhiMao avatar Sep 26 '24 23:09 HuanzhiMao

Hey @vandyxiaowei, would you mind taking a look at this PR? The Vertex AI library is a bit tricky to deal with, and I want to make sure I didn't mess up stuff. Thanks!

HuanzhiMao avatar Sep 26 '24 23:09 HuanzhiMao