self-operating-computer icon indicating copy to clipboard operation
self-operating-computer copied to clipboard

Fixes Gemini model name in config.py and improves JSON parsing success rate

Open fbader927 opened this issue 10 months ago • 2 comments

What does this PR do?

Allows Gemini to work again and reduce its failure rate.

  • Fixes Gemini model name in config.py
  • Improve success rate of Gemini tasks by fixing JSON parsing of backticks.

Requirement/Documentation

No additional requirement document. The changes align with the latest updates to the repo and functions with the updated model, along with better handling of JSON parsing.

Type of change

  • [x] Bug fix (non-breaking change which fixes an issue)

Mandatory Tasks

  • [x] Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected. Make sure before submmiting this PR you run tests with evaluate.py

fbader927 avatar Jan 23 '25 11:01 fbader927

@fbader927 thanks for the PR. Did they switch out gemini-pro-vision? Is gemini-exp-1206 the most capable one? I'll look into merging this.

joshbickett avatar Feb 19 '25 20:02 joshbickett

They did switch out 'gemini-pro-vision'. They seem to change it every 1-2 months.

'gemini-exp-1206' was the best model at the time, but there seems to be better models this month alone. Gemini has been really good for this considering how cheap it is. Also, the latest models have been more reliable with cursor movements, given it's high success rate with different tasks I've tested.

fbader927 avatar Feb 20 '25 20:02 fbader927