inference
inference copied to clipboard
Published
1 month ago
•
xorbitsai
xorbitsai
xinference 使用 qwen 模型，出现 PydanticSchemaGenerationError 错误
System Info / 系統信息
Redhat Linux 8.4 x64 系统，docker-ce-26.1.3, 使用 CPU。 模型 qwen1.5-chart, qwen2.5-instruct 参数：llama.cpp, ggufv2, 0_5, q4_k_m, N GPU Layers
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
- [x] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] installation from source / 从源码安装
Version info / 版本信息
xprobe/xinference:v1.1.1-cpu xprobe/xinference:v1.2.0-cpu
The command used to start Xinference / 用以启动 xinference 的命令
docker run --name xinference -d
-p 9997:9997
-e XINFERENCE_MODEL_SRC=modelscope
-e XINFERENCE_HOME=/root/.xinference
-v /home/opt/models/xinference:/root/.xinference
xprobe/xinference:v1.1.1-cpu
xinference-local -H 0.0.0.0 --log-level debug
Reproduction / 复现过程
1、选择 Running Models，在 qwen2.5-instruct 模型上点击 Launch WEB UI，然后输入消息，点击 submit 。
2、网页出现，could not parse server SyntaxError Unexpected token 'l', "Internal S" is not valid json 错误。
3、xinference 容器日志出现 pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set arbitrary_types_allowed=True in the model_config to ignore this error or implement __get_pydantic_core_schema__ on your type to fully support it. 错误。
Expected behavior / 期待表现
怎么避免这种错误，正常使用 qwen 模型。
docker镜像 xprobe/xinference:v1.2.1-cpu 使用 deepseek-r1-distill-qwen-pytorch-1_5b模型时遇到同样报错。 Name: pydantic Version: 2.10.6
Name: fastapi Version: 0.115.7
Name: starlette Version: 0.45.2
File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/opt/conda/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/opt/conda/lib/python3.11/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/fastapi/routing.py", line 291, in app
solved_result = await solve_dependencies(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/fastapi/dependencies/utils.py", line 666, in solve_dependencies
) = await request_body_to_args( # body_params checked above
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/fastapi/dependencies/utils.py", line 891, in request_body_to_args
fields_to_extract = get_cached_model_fields(first_field.type_)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/fastapi/_compat.py", line 659, in get_cached_model_fields
return get_model_fields(model)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/fastapi/_compat.py", line 285, in get_model_fields
return [
^
File "/opt/conda/lib/python3.11/site-packages/fastapi/_compat.py", line 286, in <listcomp>
ModelField(field_info=field_info, name=name)
File "<string>", line 6, in __init__
File "/opt/conda/lib/python3.11/site-packages/fastapi/_compat.py", line 111, in __post_init__
self._type_adapter: TypeAdapter[Any] = TypeAdapter(
^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/type_adapter.py", line 228, in __init__
self._init_core_attrs(
File "/opt/conda/lib/python3.11/site-packages/pydantic/type_adapter.py", line 290, in _init_core_attrs
core_schema = schema_generator.generate_schema(self._type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
schema = self._generate_schema_inner(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 863, in _generate_schema_inner
return self._annotated_schema(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 1977, in _annotated_schema
schema = self._apply_annotations(source_type, annotations)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 2056, in _apply_annotations
schema = get_inner_schema(source_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
schema = self._handler(source_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 2131, in new_handler
schema = metadata_get_schema(source, get_inner_schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 2127, in <lambda>
lambda source, handler: handler(source)
^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
schema = self._handler(source_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 2037, in inner_handler
schema = self._generate_schema_inner(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
return self.match_type(obj)
^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 986, in match_type
return self._match_generic_type(obj, origin)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 1014, in _match_generic_type
return self._union_schema(obj)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 1325, in _union_schema
choices.append(self.generate_schema(arg))
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
schema = self._generate_schema_inner(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
return self.match_type(obj)
^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 995, in match_type
return self._unknown_type_schema(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 513, in _unknown_type_schema
raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.
If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.
For further information visit https://errors.pydantic.dev/2.10/u/schema-for-unknown-type
