FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

stream mode error

Open day9011 opened this issue 1 year ago • 1 comments

https://github.com/THUDM/ChatGLM2-6B/issues/308 Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 261, in wrap await func() File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 250, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.10/dist-packages/fastchat/serve/openai_api_server.py", line 506, in chat_completion_stream_generator yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n" File "/usr/local/lib/python3.10/dist-packages/pydantic/main.py", line 1083, in json raise TypeError('dumps_kwargs keyword arguments are no longer supported.') TypeError: dumps_kwargs keyword arguments are no longer supported.

day9011 avatar Apr 30 '24 08:04 day9011

same error when using pydantic v2

usamimeri avatar May 17 '24 05:05 usamimeri

It should have been fixed by #3356

suquark avatar May 23 '24 21:05 suquark

It should have been fixed by #3356

Hi @suquark I just installed FastChat 0.2.36 and Pydantic 2.9.0 using pip3 install "fschat[model_worker,webui]", but I'm still unable to start the inference service for ChatGLM3-7B. The error is as follows. This opened issue comments is effective.

2024-09-10 05:47:50 | ERROR | stderr |   + Exception Group Traceback (most recent call last):
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
2024-09-10 05:47:50 | ERROR | stderr |   |     result = await app(  # type: ignore[func-returns-value]
2024-09-10 05:47:50 | ERROR | stderr |   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     return await self.app(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await super().__call__(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/applications.py", line 113, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await self.middleware_stack(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     raise exc
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await self.app(scope, receive, _send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await self.app(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-09-10 05:47:50 | ERROR | stderr |   |     raise exc
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-09-10 05:47:50 | ERROR | stderr |   |     await app(scope, receive, sender)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/routing.py", line 715, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     await self.middleware_stack(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/routing.py", line 735, in app
2024-09-10 05:47:50 | ERROR | stderr |   |     await route.handle(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
2024-09-10 05:47:50 | ERROR | stderr |   |     await self.app(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
2024-09-10 05:47:50 | ERROR | stderr |   |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-09-10 05:47:50 | ERROR | stderr |   |     raise exc
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-09-10 05:47:50 | ERROR | stderr |   |     await app(scope, receive, sender)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
2024-09-10 05:47:50 | ERROR | stderr |   |     await response(scope, receive, send)
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/responses.py", line 250, in __call__
2024-09-10 05:47:50 | ERROR | stderr |   |     async with anyio.create_task_group() as task_group:
2024-09-10 05:47:50 | ERROR | stderr |   |   File "/opt/python-3.11.9/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
2024-09-10 05:47:50 | ERROR | stderr |   |     raise BaseExceptionGroup(
2024-09-10 05:47:50 | ERROR | stderr |   | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
2024-09-10 05:47:50 | ERROR | stderr |   +-+---------------- 1 ----------------
2024-09-10 05:47:50 | ERROR | stderr |     | Traceback (most recent call last):
2024-09-10 05:47:50 | ERROR | stderr |     |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/responses.py", line 253, in wrap
2024-09-10 05:47:50 | ERROR | stderr |     |     await func()
2024-09-10 05:47:50 | ERROR | stderr |     |   File "/opt/python-3.11.9/lib/python3.11/site-packages/starlette/responses.py", line 242, in stream_response
2024-09-10 05:47:50 | ERROR | stderr |     |     async for chunk in self.body_iterator:
2024-09-10 05:47:50 | ERROR | stderr |     |   File "/opt/python-3.11.9/lib/python3.11/site-packages/fastchat/serve/openai_api_server.py", line 536, in chat_completion_stream_generator
2024-09-10 05:47:50 | ERROR | stderr |     |     yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
2024-09-10 05:47:50 | ERROR | stderr |     |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-10 05:47:50 | ERROR | stderr |     |   File "/opt/python-3.11.9/lib/python3.11/site-packages/pydantic/main.py", line 1145, in json
2024-09-10 05:47:50 | ERROR | stderr |     |     raise TypeError('`dumps_kwargs` keyword arguments are no longer supported.')
2024-09-10 05:47:50 | ERROR | stderr |     | TypeError: `dumps_kwargs` keyword arguments are no longer supported.
2024-09-10 05:47:50 | ERROR | stderr |     +------------------------------------

junruizh2021 avatar Sep 10 '24 06:09 junruizh2021