Chat.create removes "timeout" from args

[coreyb42]

Describe the bug

When making a call to the ChatCompletion create method, e.g.:

response = openai.ChatCompletion.create(
        model=self.model_id,
        messages=self.get_formatted_messages(),
        timeout=10
    )

The timeout is popd off of the kwargs by this line in the completion class (link):

timeout = kwargs.pop("timeout", None)

The pop method removes the arg from kwargs before it's passed to super, which means that the timeout doesn't have the intended effect (the local method there does do some stuff with it, but that would only allow you to increase the timeout).

To Reproduce

Run an API call with a shorter timeout:

response = openai.ChatCompletion.create(
        model=self.model_id,
        messages=self.get_formatted_messages(),
        timeout=10
    )

And observe that said timeout is not respected. The timeout remains 600 seconds.

Code snippets

No response

OS

macOS

Python version

3.10.9

Library version

openai==0.27.2

[Daedra22]

Can confirm this error, because of that, it is not possible to set a timeout.

[paramendula]

Is is true

[hallacy]

I believe you're looking for the request_timeout param (https://github.com/openai/openai-python#params). Does that solve the issue?

[ReadyPlayerEmma]

I believe you're looking for the request_timeout param (https://github.com/openai/openai-python#params). Does that solve the issue?

I was seeing this issue as well, and was able to switch to using request_timeout, after which I have not yet seen the problem. Also, reviewing the code seems to indicate that it is indeed being passed all the way down to the web request, where it should be effective. This is the second time I have encountered breaking changes between the non-chat and chat completion APIs in the Python library though (the other was engine vs model). It would be very helpful to try and keep them in sync to avoid this kind of thing.

[hallacy]

Can you describe the breaking change here? We generally try to avoid that if possible because we know how jarring of an experience it can be. That parameter is also available in our completions endpoints.

[ReadyPlayerEmma]

Can you describe the breaking change here? We generally try to avoid that if possible because we know how jarring of an experience it can be. That parameter is also available in our completions endpoints.

In the completions endpoint, there were Python examples somewhere along the line that used engine instead of model, and that still works in the completions Python API today, but not in the chat version. The same is true of the timeout parameter working fine for the standard completions, and being included in examples before, while the chat completions API does not work correctly unless you use request_timeout.

Even if the current way of doing things works fine in both places, previous examples did not always use the current parameters. Perhaps it was meant to be deprecated somewhere along the way, but there was never a point where I was using them in the Python completions API where any warnings were output indicating that was not a supported way of passing that information to the methods.

In addition, no warnings are issued when using them with the chat completions API, it just silently does not work as expected. Ideally it should not accept the incorrect and unsupported parameters and throw an error. Accepting unsupported parameters and then silently failing in unexpected ways causes a lot of unneeded headaches.

[ElieenAndBella]

I have this problem too, I get "'float' object is not callable" when I pass the param timeout, I also try the param request_timeout, it's not work.

[ReadyPlayerEmma]

I have this problem too, I get "'float' object is not callable" when I pass the param timeout, I also try the param request_timeout, it's not work.

Can you share some of your code? I think that might be a different issue.

[lucasgadams]

@ReadyPlayerEmma

The same is true of the timeout parameter working fine for the standard completions

Actually from what i see both completions and chat have the same thing, timeout is popped off and doesn't seem to do what you'd expect. But that doesn't change the fact that the docs and actual code is confusing and not clear. It seems timeout used to do a lot more (look at engine_api_resources.py) but then was part way deprecated in favor of request_timeout. But 1) it wasn't fully deprecated and 2) the docs and code are not clear. I would suggest choosing either request_timeout or timeout (or maybe change timeout to something that makes more sense to what it actually does, like retry_until or retry_timeout), and removing dead code paths like this in EngineApiResource:

            if timeout is not None:
                await obj.await_(timeout=timeout or None)

[RobertCraigie]

In our v1 beta we've unified to a single timeout argument which is passed to the client constructor and can be configured per-request using client.with_options.

from openai import OpenAI

# Configure the default for all requests:
client = OpenAI(
    # default is 60s
    timeout=20.0,
)

# More granular control:
client = OpenAI(
    timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
)

# Override per-request:
client.with_options(timeout=5 * 1000).chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": "How can I list all files in a directory using Python?",
        }
    ],
)

https://github.com/openai/openai-python/tree/v1?tab=readme-ov-file#timeouts