litellm icon indicating copy to clipboard operation
litellm copied to clipboard

[Bug]: TimeoutError -> reaches prometheus for success logging!

Open shuther opened this issue 1 year ago • 0 comments

What happened?

While I received an error from the upstream endpoint (TimeoutError), the debug statement is : reaches prometheus for success logging!

Maybe the debug statement is just wrong (I can't find in the latest source the matching rows)

Relevant log output

litellm     | 14:18:53 - LiteLLM:INFO: utils.py:1214 -
litellm     |
litellm     | POST Request Sent from LiteLLM:
litellm     | curl -X POST \
litellm     | http://linuxmain.local:11434/api/generate \
litellm     | -d '{'model': 'mistral', 'prompt': 'You are a helpful assistant.Hello!', 'options': {}, 'stream': False}'
litellm     |
litellm     |
litellm     |
litellm     | Traceback (most recent call last):
litellm     |   File "/usr/local/lib/python3.11/site-packages/litellm/llms/ollama.py", line 293, in ollama_acompletion
litellm     |     resp = await session.post(url, json=data)
litellm     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm     |   File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 601, in _request
litellm     |     await resp.start(conn)
litellm     |   File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 960, in start
litellm     |     with self._timer:
litellm     |   File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in __exit__
litellm     |     raise asyncio.TimeoutError from None
litellm     | TimeoutError
litellm     | 14:19:45 - LiteLLM:DEBUG: utils.py:1032 - Logging Details: logger_fn - None | callable(logger_fn) - False
litellm     | 14:19:46 - LiteLLM:DEBUG: utils.py:1032 - Logging Details LiteLLM-Failure Call
litellm     | 14:19:46 - LiteLLM:DEBUG: utils.py:2450 - reaches prometheus for success logging!

Twitter / LinkedIn details

No response

shuther avatar May 06 '24 14:05 shuther