[BUG] When the sse_starlette>=1.8.0, the stream output of the api is abnormal, and the stream output is normal when the sse_starlette==1.6.5.

[Samge0]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

When the sse_starlette>=1.8.0, the stream output of the api is abnormal

期望行为 | Expected Behavior

No response

复现方法 | Steps To Reproduce

When the api mode is running, it is found that the stream mode cannot streaming output normally. After comparison, it is found that the sse_starlette==1.6.5 can streaming output normally. Perhaps there are some compatibility problems with the higher version of the sse_starlette.

运行环境 | Environment

- OS:Windows 10 企业版 LTSC 21H2 19044.3693
- Python:3.10.13
- Transformers:4.32.0
- PyTorch:2.1.0+cu118
- CUDA:11.8

备注 | Anything else?

Current dependency information: (Qwen) PS F:\Space\PRO\ai\Qwen> pip list
Package Version

---

accelerate 0.24.1
aiofiles 23.2.1
aiohttp 3.9.0
aiosignal 1.3.1
altair 5.1.2
annotated-types 0.6.0
anyio 3.7.1
async-timeout 4.0.3
attrs 23.1.0
certifi 2023.11.17
charset-normalizer 3.3.2
click 8.1.7
colorama 0.4.6
contourpy 1.2.0
cycler 0.12.1
distro 1.8.0
einops 0.7.0 exceptiongroup 1.2.0 fastapi 0.104.1 ffmpy 0.3.1 filelock 3.13.1 fonttools 4.45.0 frozenlist 1.4.0 fsspec 2023.10.0 gradio 3.41.2 gradio_client 0.5.0 h11 0.14.0 httpcore 1.0.2 httpx 0.25.1 huggingface-hub 0.17.3 idna 3.4 importlib-resources 6.1.1 iniconfig 2.0.0 Jinja2 3.1.2 jsonschema 4.20.0 jsonschema-specifications 2023.11.1 kiwisolver 1.4.5 latex2mathml 3.76.0 Markdown 3.5.1 MarkupSafe 2.1.3 matplotlib 3.8.2 mdtex2html 1.2.0 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 multidict 6.0.4 networkx 3.2.1 numpy 1.26.0 openai 1.3.7 orjson 3.9.10 packaging 23.2 pandas 2.1.3 Pillow 10.1.0 pip 23.3 pluggy 1.3.0 psutil 5.9.6 pydantic 2.5.2 pydantic_core 2.14.5 pydub 0.25.1 pyparsing 3.1.1 pytest 7.4.3 python-dateutil 2.8.2 python-multipart 0.0.6 pytz 2023.3.post1 PyYAML 6.0.1 referencing 0.31.0 regex 2023.10.3 requests 2.31.0 rpds-py 0.13.1 safetensors 0.4.0 scipy 1.11.3 semantic-version 2.10.0 setuptools 68.0.0 six 1.16.0 sniffio 1.3.0 sse-starlette 1.6.5 starlette 0.27.0 sympy 1.12 tiktoken 0.5.1 tokenizers 0.13.3 tomli 2.0.1 toolz 0.12.0 torch 2.1.0+cu118 torchaudio 2.1.0+cu118 torchvision 0.16.0+cu118 tqdm 4.66.1 transformers 4.32.0 transformers-stream-generator 0.0.4 typing_extensions 4.8.0 tzdata 2023.3 urllib3 2.1.0 uvicorn 0.24.0.post1 websockets 11.0.3 wheel 0.41.2 xformers 0.0.22.post4+cu118 yarl 1.9.3