Issue with run polling(sleep)

[smbatyankh]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• [X] This is an issue with the Python library

Describe the bug

Hi OpenAI team,

I'm reaching out to report a possible issue or improvement regarding the polling mechanism in asynchronous functions.

Problem Description

The current implementation of the poll method uses the time.sleep() function, which is blocking and halts the execution of other asynchronous tasks during polling. This behavior affects concurrency, preventing other coroutines from running efficiently.

To Reproduce

---

Code snippets

import asyncio

async def poll(
    self,
    run_id: str,
    thread_id: str,
    extra_headers: dict | None = None,
    extra_query: dict | None = None,
    extra_body: dict | None = None,
    timeout: float | 'httpx.Timeout' | None = 'NOT_GIVEN',
    poll_interval_ms: int | 'NotGiven' = 'NOT_GIVEN',
) -> 'Run':
    extra_headers = {"X-Stainless-Poll-Helper": "true", **(extra_headers or {})}

    if is_given(poll_interval_ms):
        extra_headers["X-Stainless-Custom-Poll-Interval"] = str(poll_interval_ms)

    terminal_states = {"requires_action", "cancelled", "completed", "failed", "expired"}
    while True:
        response = await self.with_raw_response.retrieve(
            thread_id=thread_id,
            run_id=run_id,
            extra_headers=extra_headers,
            extra_body=extra_body,
            extra_query=extra_query,
            timeout=timeout,
        )

        run = response.parse()
        if run.status in terminal_states:
            return run

        if not is_given(poll_interval_ms):
            from_header = response.headers.get("openai-poll-after-ms")
            if from_header is not None:
                poll_interval_ms = int(from_header)
            else:
                poll_interval_ms = 1000

        await asyncio.sleep(poll_interval_ms / 1000)

OS

macos

Python version

Python v3.11.4

Library version

openai v1.26.0

[CharlyJazz]

Hey just curiosity, do you have a alternative to avoid asyncio.sleep?

[smbatyankh]

There are likely a few alternatives available, but they would involve significant changes to the implementation, such as utilizing call_later or queues. If the current implementation already includes a sleep function and there's no specific reason to use a blocking sleep here, it should be simple to switch to a non-blocking sleep.

However, I'm not sure if this was intentional or just an oversight in the implementation.

[rattrayalex]

Thanks for reporting – I confirm the problem and hope we'll have a fix out soon.

[RobertCraigie]

This will be fixed in the next release: https://github.com/openai/openai-python/pull/1414