How to use proxy-based proxy clients on google.generativeai

[Blues-star]

Description of the feature request:

In OpenAI's Python SDK, you can implement client-side proxying through the following operation https://github.com/openai/openai-python#configuring-the-http-client

import httpx
from openai import OpenAI

client = OpenAI(
    # Or use the `OPENAI_BASE_URL` env var
    base_url="http://my.test.server.example.com:8083",
    http_client=httpx.Client(
        proxies="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)

I am very curious about how I should implement the above operation in google.generativeai sdk?

What problem are you trying to solve with this feature?

I want to avoid using the system-wide global proxy as much as possible to achieve the above functionality

Any other information you'd like to share?

No response

[HawkClaws]

It is for llamaindex, but I made it. https://github.com/HawkClaws/proxy_gemini

[tianlichunhong]

没有找到prox的方法

[markmcd]

There are a couple of ways you can proxy with the SDK.

You can force the SDK to use HTTP traffic (it's gRPC by default) with genai.configure(..., transport="rest"), then the underlying requests will honour the typical environment variables - HTTP_PROXY=... or ALL_PROXY=... (note that they must be upper case).

If you have a fake endpoint or want to send the traffic to a different host directly, you can use genai.configure(client_options={'api_endpoint': 'path.to.my.domain'})

[daiaji]

If you have a fake endpoint or want to send the traffic to a different host directly, you can use genai.configure(client_options={'api_endpoint': 'path.to.my.domain'})

Are you saying that I can now use genai.configure to configure a reverse proxy for the Gemini API?

[markmcd]

Are you saying that I can now use genai.configure to configure a reverse proxy for the Gemini API?

It's not built specifically for reverse proxying, we use it for staging/preprod endpoints internally, but you do you :)