Add support for per-request metadata/headers in GenerativeModel.generate_content

[chitalian]

Description of the feature request:

Currently, the Gemini Python SDK only allows setting metadata (headers) at the client configuration level through default_metadata. This makes it difficult to set different headers for different requests, particularly when integrating with services like Helicone that require user-specific headers.

Other LLM client libraries (OpenAI, Anthropic) support setting headers on a per-request basis through parameters like extra_headers. Adding similar functionality to the Gemini SDK would improve flexibility and make it easier to integrate with proxy services.

What problem are you trying to solve with this feature?

When using the Gemini API through a proxy service like Helicone, it's important to be able to set user-specific headers (like Helicone-User-Id) on a per-request basis. Currently, this requires creating a new client instance for each user or reconfiguring the client before each request, which is inefficient and cumbersome.

Ideally, I would like to be able to do something like this:

config = {
    "messages": [
        {"role": "system", "content": system_msg},
        {"role": "user", "content": human_msg},
    ],
    "response_model": output_structure,
}

user_id = user_id_var.get()
if user_id:
    config["extra_headers"] = {}
    config["extra_headers"]["Helicone-User-Id"] = user_id

response = client.chat.completions.create(**config)

This would allow setting different headers for different requests without having to reconfigure the client or create multiple client instances.

Any other information you'd like to share?

Proposed implementation:

The GenerativeModel.generate_content method could be modified to accept an additional parameter like metadata or extra_headers that would be passed to the underlying client method. This would allow users to set headers on a per-request basis while maintaining backward compatibility.

Example implementation:

def generate_content(
    self,
    contents: content_types.ContentsType,
    *,
    generation_config: generation_types.GenerationConfigType | None = None,
    safety_settings: safety_types.SafetySettingOptions | None = None,
    stream: bool = False,
    tools: content_types.FunctionLibraryType | None = None,
    tool_config: content_types.ToolConfigType | None = None,
    request_options: helper_types.RequestOptionsType | None = None,
    extra_headers: dict[str, str] | None = None,  # New parameter
) -> generation_types.GenerateContentResponse:
    # ... existing code ...
    
    if request_options is None:
        request_options = {}
    
    # Convert extra_headers to metadata format if provided
    metadata = []
    if extra_headers:
        metadata = [(k, v) for k, v in extra_headers.items()]
    
    # Add metadata to request_options
    if metadata:
        request_options["metadata"] = metadata
    
    # ... rest of the method ...

This would provide a more consistent experience across different LLM client libraries and make it easier to integrate with proxy services.

[julurisaichandu]

I would like to work on this @chitalian , could you assign me this?

[mahimairaja]

Hi, I would like to work on this issue

[mahimairaja]

@MarkDaoust Can you please assign this issue to me

[julurisaichandu]

can we just work on the issue without getting assigned @mahimairaja @MarkDaoust? because I already see a PR from someone for this issue and the issue wasn't even assigned to them

[MarkDaoust]

https://github.com/googleapis/python-genai

This SDK is deprecated, we're really not prioritizing the addition of new features.

Consider switching to the new SDK which already has this feature.

https://github.com/googleapis/python-genai/blob/4a72626a9be25ce8973d25b32cc69db49cde0dfb/google/genai/types.py#L1112-L1134

[github-actions[bot]]

Marking this issue as stale since it has been open for 14 days with no activity. This issue will be closed if no further activity occurs.

[github-actions[bot]]

This issue was closed because it has been inactive for 28 days. Please post a new issue if you need further assistance. Thanks!