[Bug] 高并发下请求异常

[czcl]

Checklist

• [ ] 1. I have searched related issues but cannot get the expected help.
• [ ] 2. The bug has not been fixed in the latest version.
• [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

asyncio.exceptions.InvalidStateError: invalid state Exception in callback <bound method StreamingSemaphore.release of <lmdeploy.turbomind.turbomind.StreamingSemaphore object at 0x7fc9be8a94e0>> handle: <Handle StreamingSemaphore.release> Traceback (most recent call last): File "uvloop/cbhandles.pyx", line 61, in uvloop.loop.Handle._run File "/usr/local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 415, in release self.fut.set_result(None) asyncio.exceptions.InvalidStateError: invalid state

Reproduction

CUDA_VISIBLE_DEVICES=3 lmdeploy serve api_server /data/InternVL3-78B-AWQ --server-port 23334 --cache-max-entry-count 0.1 --session-len 5000

Environment

python 3.10

Error traceback

[czcl]

并发测试脚本： 循环100次，每次都发送requests.post 就会复现 请求内容： request_data = { "id": rid, "model": "/mnt/InternVL3-78B-AWQ", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "描述图片内容" }, { "type": "image_url", "image_url": { "url": "图片url" } } ] } ], "temperature": 0.7, "stream": True }

text和图片url都是不同的

[shell-nlp]

同问

[czcl]

你好当我看到的时候我会自动回复您

[huu3301]

0.9.0版本偶现同样的问题 qwen-1 | 2025-07-02 19:34:16,159 - lmdeploy - INFO - turbomind.py:711 - [async_stream_infer] CancelledError qwen-1 | 2025-07-02 19:34:16,159 - lmdeploy - ERROR - async_engine.py:599 - [safe_run] exception caught: CancelledError Cancelled by cancel scope 78f6ef7123f0 qwen-1 | [TM][INFO] [Interrupt] slot 0, request 691, stop 1, end 0 qwen-1 | Exception in callback <bound method StreamingSemaphore.release of <lmdeploy.turbomind.turbomind.StreamingSemaphore object at 0x78f6ee055430>> qwen-1 | handle: <Handle StreamingSemaphore.release> qwen-1 | Traceback (most recent call last): qwen-1 | File "uvloop/cbhandles.pyx", line 61, in uvloop.loop.Handle._run qwen-1 | File "/usr/local/lib/python3.12/dist-packages/lmdeploy/turbomind/turbomind.py", line 488, in release qwen-1 | self.fut.set_result(None) qwen-1 | asyncio.exceptions.InvalidStateError: invalid state

[gengchaogit]

最新版本v0.10.2也有此问题

当输入达到32K的下一个阶段64K左右就会报错 4卡v100运行30b 32b好像都会报这个错误，还有gpt oss 20b

2025-11-24 09:04:56,205 - lmdeploy - ERROR - async_engine.py:710 - [safe_run] exception caught: CancelledError Cancelled via cancel scope 7f071f010860 by <Task pending name='Task-56' coro=<RequestResponseCycle.run_asgi() running at /root/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py:403> cb=[set.discard()]> 2025-11-24 09:04:56,205 - lmdeploy - ERROR - async_engine.py:695 - [model_inst] exception caught: Cancelled via cancel scope 7f071f010860 by <Task pending name='Task-56' coro=<RequestResponseCycle.run_asgi() running at /root/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py:403> cb=[set.discard()]> Exception in callback StreamingSemaphore.release() handle: <Handle StreamingSemaphore.release()> Traceback (most recent call last): File "/root/miniconda3/envs/lmdeploy/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/root/miniconda3/envs/lmdeploy/lib/python3.12/site-packages/lmdeploy/turbomind/turbomind.py", line 513, in release self.fut.set_result(None) asyncio.exceptions.InvalidStateError: invalid state

[czcl]

你好当我看到的时候我会自动回复您