[Feature]: Grounding on Google AI Studio

[traderpedroso]

The Feature

Dear LiteLLM Team, I am writing to request the addition of support for grounding on Google Web Search when interacting with the Gemini 2.0 Flash model through LiteLLM. This functionality is now available via the v1alpha and v1beta on Google AI Studio endpoint of the Generative Language API: https://generativelanguage.googleapis.com/v1alpha/models/gemini-2.0-flash-exp.

{

  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "best stock for 2025"
        }
      ]
    }
  ],
  "systemInstruction": {
    "role": "user",
    "parts": [
      {
        "text": "Searching on google, answer"
      }
    ]
  },
  "tools": [
    {
      "googleSearch": {}
    }
  ],
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_CIVIC_INTEGRITY",
      "threshold": "BLOCK_NONE"
    }
  ]
}

Motivation, pitch

Currently, LiteLLM provides a fantastic abstraction layer for accessing various LLMs, including Gemini models. However, the ability to ground responses with real-time information from the web is crucial for many applications, particularly those requiring: Up-to-date Information: LLMs have a knowledge cutoff, and grounding with web search allows them to access the most recent data, ensuring more accurate and relevant responses. Fact Verification: Grounding enables LLMs to verify the information they generate, reducing the risk of hallucinations and providing users with more trustworthy outputs. Enhanced Contextual Understanding: By leveraging external information, LLMs can better understand the nuances of a user's query and provide more comprehensive and context-aware answers. Real-World Applications: Many use cases, such as research, content creation, and customer support, require access to external information, making grounding an essential feature.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response

[antonkulaga]

so is there any way to activate gemini-based google search now? @traderpedroso

[krrishdholakia]

I believe this should already work - https://docs.litellm.ai/docs/providers/vertex#grounding---web-search

from litellm import completion 
import os

os.environ["GEMINI_API_KEY"] = "" 

tools = [{"googleSearch": {}}] # 👈 ADD GOOGLE SEARCH

resp = litellm.completion(
                    model="gemini/gemini-1.5-flash",
                    messages=[{"role": "user", "content": "Who won the world cup?"}],
                    tools=tools,
                )

print(resp)

[zenjerr]

Tried the https://docs.litellm.ai/docs/providers/vertex#grounding---web-search , and it works.

How about grounding with user data from a GCP datastore? Either through LiteLLM SDK or LiteLLM proxy server.

[d1egoaz]

Example of the proxy config to always enable Grounding with Google search (and reasoning)

  - model_name: Google/gemini-2.5-flash-with-search
    litellm_params:
      model: gemini/gemini-2.5-flash-preview-05-20
      api_key: os.environ/GEMINI_API_KEY
      merge_reasoning_content_in_choices: true
      allowed_openai_params: ["reasoning_effort"]
      reasoning_effort: "low"
      tools:
        - googleSearch: {}
      cache_control_injection_points:
        - location: message

[krrishdholakia]

grounding with user data from a GCP datastore?

hey @zenjerr not sure i've come across this, can you. point me to any docs on this?

It sounds like our bedrock knowledge bases implementation - https://docs.litellm.ai/docs/providers/bedrock_vector_store