Langchain passes `prompt` as a `list` instead of `str`

[beratcmn]

As mentioned in the title this simple example passes a list instead of a str.

Raw request:

Error Message: INFO: 127.0.0.1:44226 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ return await self.app(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 282, in __call__ await super().__call__(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__ await self.middleware_stack(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__ raise exc File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__ await self.app(scope, receive, _send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__ await self.app(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ raise exc File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ await self.app(scope, receive, sender) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__ raise e File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__ await self.app(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__ await route.handle(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 241, in app raw_response = await run_endpoint_function( File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 167, in run_endpoint_function return await dependant.call(**values) File "/home/aipath/repo/local-agent/.venv/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 106, in create_completion request = CompletionRequest(**await raw_request.json()) File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for CompletionRequest prompt str type expected (type=type_error.str)

[zhuohan123]

Thanks for reporting this issue. Our current OpenAI frontend only supports passing a single-string prompt. We will add support for other formats in the future. Meanwhile, feel free to modify vllm/entrypoints/openai/api_server.py to support this and your PR is greatly appreciated!

[zhuohan123]

Please refer to https://python.langchain.com/docs/integrations/llms/vllm for the latest langchain integration.