FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

[embedding model & connect with Langchain] "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error

Open panxnan opened this issue 1 year ago • 0 comments

current fastchat version: 0.2.36

I served the embedding model in openai style with the code below and I want to connet with LangChain, but errors occured! how to deal with this problem?

-- serve embedding model comand

CUDA_VISIBLE_DEVICES=2 ~/.conda/envs/fastchat/bin/python \
    -m fastchat.serve.model_worker \
    --model-path /public/llm/models/bge-large-zh-v1.5 \
    --model-names "alias-embeddings,gpt-3.5-turbo,text-davinci-003,text-embedding-ada-002,bge-large-zh-v1.5"\
    --port 31002 \
    --worker http://localhost:31002 &

-- LangChain

from langchain.prompts.example_selector import SemanticSimilarityExampleSelector
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings


a = OpenAIEmbeddings(
    model='text-embedding-ada-002'
)
a.embed_documents(['hi there', '你好'])

-- Error

2024-03-29 16:27:01 | INFO | stdout | INFO:     127.0.0.1:58774 - "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error
2024-03-29 16:27:01 | ERROR | stderr | ERROR:    Exception in ASGI application
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
2024-03-29 16:27:01 | ERROR | stderr |     sock = connection.create_connection(
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
2024-03-29 16:27:01 | ERROR | stderr |     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/socket.py", line 962, in getaddrinfo
2024-03-29 16:27:01 | ERROR | stderr |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | socket.gaierror: [Errno -2] Name or service not known
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
2024-03-29 16:27:01 | ERROR | stderr |     response = self._make_request(
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
2024-03-29 16:27:01 | ERROR | stderr |     raise new_e
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
2024-03-29 16:27:01 | ERROR | stderr |     self._validate_conn(conn)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
2024-03-29 16:27:01 | ERROR | stderr |     conn.connect()
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 611, in connect
2024-03-29 16:27:01 | ERROR | stderr |     self.sock = sock = self._new_conn()
2024-03-29 16:27:01 | ERROR | stderr |                        ^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connection.py", line 210, in _new_conn
2024-03-29 16:27:01 | ERROR | stderr |     raise NameResolutionError(self.host, self, e) from e
2024-03-29 16:27:01 | ERROR | stderr | urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7fb000d479d0>: Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
2024-03-29 16:27:01 | ERROR | stderr |     resp = conn.urlopen(
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
2024-03-29 16:27:01 | ERROR | stderr |     retries = retries.increment(
2024-03-29 16:27:01 | ERROR | stderr |               ^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
2024-03-29 16:27:01 | ERROR | stderr |     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
2024-03-29 16:27:01 | ERROR | stderr |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fb000d479d0>: Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)"))
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | During handling of the above exception, another exception occurred:
2024-03-29 16:27:01 | ERROR | stderr | 
2024-03-29 16:27:01 | ERROR | stderr | Traceback (most recent call last):
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
2024-03-29 16:27:01 | ERROR | stderr |     result = await app(  # type: ignore[func-returns-value]
2024-03-29 16:27:01 | ERROR | stderr |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     return await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await super().__call__(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await self.middleware_stack(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     raise exc
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await self.app(scope, receive, _send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr |     raise exc
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr |     await app(scope, receive, sender)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
2024-03-29 16:27:01 | ERROR | stderr |     await self.middleware_stack(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
2024-03-29 16:27:01 | ERROR | stderr |     await route.handle(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
2024-03-29 16:27:01 | ERROR | stderr |     await self.app(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
2024-03-29 16:27:01 | ERROR | stderr |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr |     raise exc
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-03-29 16:27:01 | ERROR | stderr |     await app(scope, receive, sender)
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
2024-03-29 16:27:01 | ERROR | stderr |     response = await func(request)
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
2024-03-29 16:27:01 | ERROR | stderr |     raw_response = await run_endpoint_function(
2024-03-29 16:27:01 | ERROR | stderr |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
2024-03-29 16:27:01 | ERROR | stderr |     return await dependant.call(**values)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/fastchat/serve/openai_api_server.py", line 717, in create_embeddings
2024-03-29 16:27:01 | ERROR | stderr |     request.input = process_input(request.model, request.input)
2024-03-29 16:27:01 | ERROR | stderr |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/fastchat/serve/openai_api_server.py", line 245, in process_input
2024-03-29 16:27:01 | ERROR | stderr |     decoding = tiktoken.model.encoding_for_model(model_name)
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/model.py", line 101, in encoding_for_model
2024-03-29 16:27:01 | ERROR | stderr |     return get_encoding(encoding_name_for_model(model_name))
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/registry.py", line 73, in get_encoding
2024-03-29 16:27:01 | ERROR | stderr |     enc = Encoding(**constructor())
2024-03-29 16:27:01 | ERROR | stderr |                      ^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken_ext/openai_public.py", line 72, in cl100k_base
2024-03-29 16:27:01 | ERROR | stderr |     mergeable_ranks = load_tiktoken_bpe(
2024-03-29 16:27:01 | ERROR | stderr |                       ^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 147, in load_tiktoken_bpe
2024-03-29 16:27:01 | ERROR | stderr |     contents = read_file_cached(tiktoken_bpe_file, expected_hash)
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 64, in read_file_cached
2024-03-29 16:27:01 | ERROR | stderr |     contents = read_file(blobpath)
2024-03-29 16:27:01 | ERROR | stderr |                ^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.local/lib/python3.11/site-packages/tiktoken/load.py", line 25, in read_file
2024-03-29 16:27:01 | ERROR | stderr |     resp = requests.get(blobpath)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/api.py", line 73, in get
2024-03-29 16:27:01 | ERROR | stderr |     return request("get", url, params=params, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/api.py", line 59, in request
2024-03-29 16:27:01 | ERROR | stderr |     return session.request(method=method, url=url, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
2024-03-29 16:27:01 | ERROR | stderr |     resp = self.send(prep, **send_kwargs)
2024-03-29 16:27:01 | ERROR | stderr |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
2024-03-29 16:27:01 | ERROR | stderr |     r = adapter.send(request, **kwargs)
2024-03-29 16:27:01 | ERROR | stderr |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-29 16:27:01 | ERROR | stderr |   File "/home/panxingnan/.conda/envs/fastchat/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
2024-03-29 16:27:01 | ERROR | stderr |     raise ConnectionError(e, request=request)
2024-03-29 16:27:01 | ERROR | stderr | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fb000d479d0>: Failed to resolve 'openaipublic.blob.core.windows.net' ([Errno -2] Name or service not known)"))

panxnan avatar Mar 29 '24 09:03 panxnan