feedback icon indicating copy to clipboard operation
feedback copied to clipboard

[gemini-function-calling]: Unexpected response from given prompt

Open xprilion opened this issue 1 year ago • 1 comments

The prompt provided in Step 8. Generate a function call might at times provide responses which ask for when the exchange was made. For instance:

Screenshot 2024-03-16 at 12 25 46 AM

^ Using "gemini-1.0-pro-001"

The better way to frame this prompt would be:

prompt = """What is the latest exchange rate from Australian dollars to Swedish krona?
How much is 500 Australian dollars worth in Swedish krona?"""

this accurately invokes the expected function calling.

xprilion avatar Mar 15 '24 18:03 xprilion

Hi @xprilion, apologies that we missed this feedback. And thank you for reporting this behavior!

  • I intentionally crafted the prompt this way to exercise the following parameter description in the FunctionDeclaration, which should fall back to the value latest if a date is not specified:
            "currency_date": {
                "type": "string",
                "description": "A date that must always be in YYYY-MM-DD format or the value 'latest' if a time period is not specified"
            },
    
  • I've updated the codelab to use gemini-1.5-pro-001 (after testing it and validating that it is behaving as expected) and it's improved reasoning and Function Calling behavior. This should increase the level of determinism and improve the behavior here so that the model follows the instructions in the parameter description as intended.
  • Finally, I've updated the feedback link in the codelab to auto assign folks so we don't miss feedback like this in the future!

koverholt avatar May 24 '24 22:05 koverholt