app Dockerfile: Add the possibility to add proxy when lauching container via ENV VAR

I'm behind a corporate proxy and I've setted the proxy where i can. But when I'm trying to use

import os

from langchain.embeddings import OpenAIEmbeddings

os.environ["OPENAI_API_KEY"] = "rassndosssms-sssstrissnsg"

embeddings = OpenAIEmbeddings(openai_api_base="http://localhost:8444/v1")
text = "Prem is an easy to use open source AI platform."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text],1)

I'm receiving the error

Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised APIError: Invalid response object from API: 'Traceback (most recent call last):\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn\n    conn = connection.create_connection(\n  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection\n    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):\n  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo\n    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\nsocket.gaierror: [Errno -3] Temporary failure in name resolution\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 714, in urlopen\n    httplib_response = self._make_request(\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 403, in _make_request\n    self._validate_conn(conn)\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn\n    conn.connect()\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect\n    self.sock = conn = self._new_conn()\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn\n    raise NewConnectionError(\nurllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send\n    resp = conn.urlopen(\n  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 798, in urlopen\n    retries = retries.increment(\n  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment\n    raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=\'openaipublic.blob.core.windows.net\', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\'))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__\n    await self.app(scope, receive, _send)\n  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__\n    await self.app(scope, receive, send)\n  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__\n    raise exc\n  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__\n    await self.app(scope, receive, sender)\n  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__\n    raise e\n  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__\n    await self.app(scope, receive, send)\n  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__\n    await route.handle(scope, receive, send)\n  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle\n    await self.app(scope, receive, send)\n  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app\n    response = await func(request)\n  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app\n    raw_response = await run_endpoint_function(\n  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function\n    return await dependant.call(**values)\n  File "/usr/src/app/routes.py", line 61, in embeddings_openai\n    encoding = tiktoken.model.encoding_for_model("text-embedding-ada-002")\n  File "/usr/local/lib/python3.10/site-packages/tiktoken/model.py", line 75, in encoding_for_model\n    return get_encoding(encoding_name)\n  File "/usr/local/lib/python3.10/site-packages/tiktoken/registry.py", line 63, in get_encoding\n    enc = Encoding(**constructor())\n  File "/usr/local/lib/python3.10/site-packages/tiktoken_ext/openai_public.py", line 64, in cl100k_base\n    mergeable_ranks = load_tiktoken_bpe(\n  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 116, in load_tiktoken_bpe\n    contents = read_file_cached(tiktoken_bpe_file)\n  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 48, in read_file_cached\n    contents = read_file(blobpath)\n  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 24, in read_file\n    resp = requests.get(blobpath)\n  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get\n    return request("get", url, params=params, **kwargs)\n  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request\n    return session.request(method=method, url=url, **kwargs)\n  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request\n    resp = self.send(prep, **send_kwargs)\n  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send\n    r = adapter.send(request, **kwargs)\n  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send\n    raise ConnectionError(e, request=request)\nrequests.exceptions.ConnectionError: HTTPSConnectionPool(host=\'openaipublic.blob.core.windows.net\', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\'))\n' (HTTP response code was 500).

This error is generated by ghcr.io/premai-io/embeddings-all-minilm-l6-v2-cpu:1.0.2 which is started without proxy configuration and is not able to go online.

Can a proxy be added in setting and be concatenated as environment variable when launchinh these dockers?

Jul 13 '23 15:07 cesco69

Everything runs locally both Prem Daemon and Services, so weird you having problems, and you should not require a proxy as you calling all in localhost. Am I wrong?

Jul 17 '23 09:07 tiero

Ciao @tiero ! At some point into the code, there is a call at openaipublic.blob.core.windows.net

File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=\'openaipublic.blob.core.windows.net\', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\'))
' (HTTP response code was 500).

this is the full stack trace:

Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised APIError: Invalid response object from API: 'Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 403, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 798, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=\'openaipublic.blob.core.windows.net\', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/src/app/routes.py", line 61, in embeddings_openai
    encoding = tiktoken.model.encoding_for_model("text-embedding-ada-002")
  File "/usr/local/lib/python3.10/site-packages/tiktoken/model.py", line 75, in encoding_for_model
    return get_encoding(encoding_name)
  File "/usr/local/lib/python3.10/site-packages/tiktoken/registry.py", line 63, in get_encoding
    enc = Encoding(**constructor())
  File "/usr/local/lib/python3.10/site-packages/tiktoken_ext/openai_public.py", line 64, in cl100k_base
    mergeable_ranks = load_tiktoken_bpe(
  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 116, in load_tiktoken_bpe
    contents = read_file_cached(tiktoken_bpe_file)
  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 48, in read_file_cached
    contents = read_file(blobpath)
  File "/usr/local/lib/python3.10/site-packages/tiktoken/load.py", line 24, in read_file
    resp = requests.get(blobpath)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=\'openaipublic.blob.core.windows.net\', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x7f1db722e260>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution\'))
' (HTTP response code was 500).

Jul 17 '23 12:07 cesco69

It looks like langchain related to me. Any hint @filopedraz ?

Jul 17 '23 16:07 tiero

Do you still get the error? Do you have an internet connection? Seems where the tiktoken library fetches some meta info.

Jul 17 '23 19:07 filopedraz

@filopedraz I'm behind a corporate proxy without direct internet connection. Yep, the tiktoken library fetches some meta info. I've found a workaround rebuilding the docker image of prem-daemon and set manually below there these lines .

env_variables.append("http_proxy=http://myproxy:3128")
env_variables.append("https_proxy=http://myproxy:3128")
env_variables.append("no_proxy=localhost,127.0.0.1")

But I have already system environment variables ignored by tiktoken. I think that prem-daemon launched docker does not extend custom enviroment variables to its launched containers

Jul 18 '23 06:07 cesco69

Yep, if I understood correctly, you will need to re-run the containers.

Jul 18 '23 08:07 filopedraz

Yes, to make the workaround works it's needed to rebuild prem-daemon with the above changes, relaunch docker-compose and then relaunch containers (in this case ghcr.io/premai-io/embeddings-all-minilm-l6-v2-cpu:1.0.2). Now it finds the proxy and is able to make the call to openaipublic.blob.core.windows.net .

I don't know if it could be useful a feature for configuring proxy conf or if this issue is enough.

I've solved my problems with the above workaround

Jul 18 '23 09:07 cesco69

Yes, good point, thanks. What we can do is support those ENV VAR in the official prem daemon image, so you can pass any value at runtime.

Jul 18 '23 14:07 tiero