jeromeroussin
jeromeroussin
I am making calls like this to the proxy with openai version < 1.0: ```python embeddings = await openai.Embedding.acreate(model=model, input=['random sentence here'], user='foo') ```
I am also interested in this
Here is the timeout stacktrace for one of those 6000s (non-streaming) timeouts if that helps: ``` Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions yield File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py",...
Right, a few words missing. I edited the wording.
Addressed in a later bug report: https://github.com/BerriAI/litellm/issues/7001
The issue still exists in 1.57.3. The "timeout" value of our models is not being respected and we're seeing timeouts at the default 6000s value
A stacktrace of a 6000s timeout: ``` Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions yield File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 377, in handle_async_request resp = await self._pool.handle_async_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...
It is stacktrace for gpt-4o
I second the ask for callbacks to be supported in Assistant related routes. Observability is the second half of the value of litellm for us (Authentication being the other half)....
@krrishdholakia The ask now is to extend https://github.com/BerriAI/litellm/pull/16643 and support azure as a provider